[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:50:04 dist-test-slave-vk40 krb5kdc[15310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:50:04 dist-test-slave-vk40 krb5kdc[15310](info): set up 2 sockets
Sep 28 14:50:04 dist-test-slave-vk40 krb5kdc[15310](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260928 14:50:05.875583 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:50:05 dist-test-slave-vk40 krb5kdc[15310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:50:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:50:06.044494 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46591
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:46591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:06.403079 15323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:06.403445 15323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:06.403604 15323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:06.412225 15323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:06.412424 15323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:06.412528 15323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:06.412631 15323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:06.412708 15323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:06.423943 15323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:46591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:06.425408 15323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:06.426853 15323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:06.440274 15328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:06.440191 15329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:06.440330 15331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:06.440750 15323 server_base.cc:1034] running on GCE node
I20260928 14:50:06.441583 15323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:06.443152 15323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:06.444372 15323 hybrid_clock.cc:648] HybridClock initialized: now 1790607006444320 us; error 49 us; skew 500 ppm
I20260928 14:50:06.444726 15323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:06 dist-test-slave-vk40 krb5kdc[15310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607006, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:06.451256 15323 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:50:06.453084 15323 webserver.cc:460] Webserver started at http://127.14.241.254:46605/ using document root <none> and password file <none>
I20260928 14:50:06.453810 15323 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:06.453933 15323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:06.454224 15323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:06.457402 15323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "3757269b33c84bd1a8785cc0f75ec047"
format_stamp: "Formatted at 2026-09-28 14:50:06 on dist-test-slave-vk40"
server_key: "e8532086f73152d78fb7a344570b4f7c"
server_key_iv: "c62bb4ec6db3f741b073f2eb70048db3"
server_key_version: "encryptionkey@0"
I20260928 14:50:06.458385 15323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "3757269b33c84bd1a8785cc0f75ec047"
format_stamp: "Formatted at 2026-09-28 14:50:06 on dist-test-slave-vk40"
server_key: "e8532086f73152d78fb7a344570b4f7c"
server_key_iv: "c62bb4ec6db3f741b073f2eb70048db3"
server_key_version: "encryptionkey@0"
I20260928 14:50:06.466053 15323 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260928 14:50:06.471459 15338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:06.473294 15323 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260928 14:50:06.473567 15323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "3757269b33c84bd1a8785cc0f75ec047"
format_stamp: "Formatted at 2026-09-28 14:50:06 on dist-test-slave-vk40"
server_key: "e8532086f73152d78fb7a344570b4f7c"
server_key_iv: "c62bb4ec6db3f741b073f2eb70048db3"
server_key_version: "encryptionkey@0"
I20260928 14:50:06.473871 15323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:06.496186 15323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:06.502599 15323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:06.503010 15323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:06.546950 15323 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:46591
I20260928 14:50:06.547039 15389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:46591 every 8 connection(s)
I20260928 14:50:06.548782 15323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:06.552335 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 15323
I20260928 14:50:06.552762 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:06.554265 15390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:06.555637 15303 external_mini_cluster.cc:1428] Setting key c2790aacdd1b78fda59d896e7d216556
I20260928 14:50:06.573112 15390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047: Bootstrap starting.
I20260928 14:50:06.579383 15390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:06.581625 15390 log.cc:824] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:06.587100 15390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047: No bootstrap required, opened a new log
I20260928 14:50:06.602443 15390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3757269b33c84bd1a8785cc0f75ec047" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46591 } }
I20260928 14:50:06.603236 15390 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:06.603417 15390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3757269b33c84bd1a8785cc0f75ec047, State: Initialized, Role: FOLLOWER
I20260928 14:50:06.604436 15390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3757269b33c84bd1a8785cc0f75ec047" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46591 } }
I20260928 14:50:06.605017 15390 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:06.605297 15390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:06.605605 15390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [term 0 FOLLOWER]: Advancing to term 1
Sep 28 14:50:06 dist-test-slave-vk40 krb5kdc[15310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:06.610021 15390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3757269b33c84bd1a8785cc0f75ec047" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46591 } }
I20260928 14:50:06.610911 15390 leader_election.cc:302] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3757269b33c84bd1a8785cc0f75ec047; no voters:
I20260928 14:50:06.612262 15390 leader_election.cc:288] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:06.612566 15395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:06.616998 15395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [term 1 LEADER]: Becoming Leader. State: Replica: 3757269b33c84bd1a8785cc0f75ec047, State: Running, Role: LEADER
I20260928 14:50:06.618173 15390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:06.618067 15395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3757269b33c84bd1a8785cc0f75ec047" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46591 } }
I20260928 14:50:06.626317 15392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:06.560738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.14.241.254:46591)
0928 14:50:06.562743 (+ 2005us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:06.562761 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:06.565221 (+ 2460us) server_negotiation.cc:407] Connection header received
0928 14:50:06.583649 (+ 18428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:06.583688 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:06.584417 (+ 729us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:06.584860 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:06.590709 (+ 5849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:06.592798 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:06.596337 (+ 3539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:06.597490 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:06.597678 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:06.607822 (+ 10144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:06.607837 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:06.607867 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:06.607959 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:06.617418 (+ 9459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:06.619113 (+ 1695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:06.619123 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:06.619134 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:06.619495 (+ 361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:06.620399 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:06.620406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:06.620410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:06.621498 (+ 1088us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:06.621982 (+ 484us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:06.623770 (+ 1788us) server_negotiation.cc:299] Negotiation successful
0928 14:50:06.624339 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1642,"thread_start_us":1057,"threads_started":1}
I20260928 14:50:06.627573 15397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3757269b33c84bd1a8785cc0f75ec047. Latest consensus state: current_term: 1 leader_uuid: "3757269b33c84bd1a8785cc0f75ec047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3757269b33c84bd1a8785cc0f75ec047" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46591 } } }
I20260928 14:50:06.627583 15396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3757269b33c84bd1a8785cc0f75ec047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3757269b33c84bd1a8785cc0f75ec047" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46591 } } }
I20260928 14:50:06.628343 15396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:06.628067 15397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047 [sys.catalog]: This master's current role is: LEADER
W20260928 14:50:06.640892 15410 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:50:06.641666 15410 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:50:06.643157 15403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:06.646057 15403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:06.671810 15403 catalog_manager.cc:1347] Generated new cluster ID: 0e1231090f654287bac05d58f6369376
I20260928 14:50:06.671955 15403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:06.690613 15403 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:06.692601 15403 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:06.703323 15403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3757269b33c84bd1a8785cc0f75ec047: Generated new TSK 0
I20260928 14:50:06.704242 15403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:50:06.753147 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46591
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:07.168032 15416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:07.168381 15416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:07.168535 15416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:07.178088 15416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:07.178226 15416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:07.178381 15416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:07.191201 15416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46591
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:07.193010 15416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:07.194403 15416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:07.207877 15421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:07.211913 15422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:07.213469 15424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:07.213898 15416 server_base.cc:1034] running on GCE node
I20260928 14:50:07.214614 15416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:07.215587 15416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:07.216786 15416 hybrid_clock.cc:648] HybridClock initialized: now 1790607007216719 us; error 61 us; skew 500 ppm
I20260928 14:50:07.217105 15416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:07 dist-test-slave-vk40 krb5kdc[15310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607007, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:07.226658 15416 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:50:07.228415 15416 webserver.cc:460] Webserver started at http://127.14.241.193:37563/ using document root <none> and password file <none>
I20260928 14:50:07.229175 15416 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:07.229285 15416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:07.229593 15416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:07.232868 15416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "3a128f9f176b44fa9b761b8e4447e196"
format_stamp: "Formatted at 2026-09-28 14:50:07 on dist-test-slave-vk40"
server_key: "af4f7dc69bae884ae9810f3bc90af8f7"
server_key_iv: "dd7f9fc830889884779f0c4f9c225100"
server_key_version: "encryptionkey@0"
I20260928 14:50:07.233639 15416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "3a128f9f176b44fa9b761b8e4447e196"
format_stamp: "Formatted at 2026-09-28 14:50:07 on dist-test-slave-vk40"
server_key: "af4f7dc69bae884ae9810f3bc90af8f7"
server_key_iv: "dd7f9fc830889884779f0c4f9c225100"
server_key_version: "encryptionkey@0"
I20260928 14:50:07.240324 15416 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260928 14:50:07.245229 15431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:07.246647 15416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260928 14:50:07.246826 15416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "3a128f9f176b44fa9b761b8e4447e196"
format_stamp: "Formatted at 2026-09-28 14:50:07 on dist-test-slave-vk40"
server_key: "af4f7dc69bae884ae9810f3bc90af8f7"
server_key_iv: "dd7f9fc830889884779f0c4f9c225100"
server_key_version: "encryptionkey@0"
I20260928 14:50:07.247057 15416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:07.259286 15416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:07.264935 15416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:07.265219 15416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:07.266428 15416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:07.268115 15416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:07.268198 15416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:07.268380 15416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:07.268463 15416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260928 14:50:07.324851 15416 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:45073
I20260928 14:50:07.324846 15543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:45073 every 8 connection(s)
I20260928 14:50:07.326510 15416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:07.333060 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 15416
I20260928 14:50:07.333252 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:50:07.333804 15303 external_mini_cluster.cc:1428] Setting key 856557ecb184a260c3ab2511e320d2dd
Sep 28 14:50:07 dist-test-slave-vk40 krb5kdc[15310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607007, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:07.353343 15546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:07.329838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:59219 (local address 127.14.241.254:46591)
0928 14:50:07.330970 (+ 1132us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:07.330975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:07.332617 (+ 1642us) server_negotiation.cc:407] Connection header received
0928 14:50:07.336058 (+ 3441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:07.336063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:07.336144 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:07.336236 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:07.338669 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:07.339388 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:07.342010 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:07.342436 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:07.342551 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:07.347041 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:07.347046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:07.347049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:07.347084 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:07.349876 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:07.350760 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:07.350765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:07.350768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:07.350906 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:07.351395 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:07.351399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:07.351401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:07.351676 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:07.351804 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:07.352645 (+ 841us) server_negotiation.cc:299] Negotiation successful
0928 14:50:07.352890 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":988,"thread_start_us":869,"threads_started":1}
I20260928 14:50:07.354276 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:07.330796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46591 (local address 127.14.241.193:59219)
0928 14:50:07.332459 (+ 1663us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:07.332505 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:07.335776 (+ 3271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:07.336558 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:07.336567 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:07.337408 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:07.338479 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:07.338504 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:07.339551 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:07.339555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:07.341811 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:07.341827 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:07.342805 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:07.342813 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:07.343138 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:07.343999 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:07.344022 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:07.346803 (+ 2781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:07.350071 (+ 3268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:07.350079 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:07.350095 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:07.350622 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:07.351014 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:07.351019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:07.351022 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:07.351259 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:07.351808 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:07.351817 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:07.352200 (+ 383us) client_negotiation.cc:769] Sending connection context
0928 14:50:07.352514 (+ 314us) client_negotiation.cc:240] Negotiation successful
0928 14:50:07.353114 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1340,"thread_start_us":914,"threads_started":1}
I20260928 14:50:07.355813 15544 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46591
I20260928 14:50:07.356127 15544 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:07.356997 15544 heartbeater.cc:511] Master 127.14.241.254:46591 requested a full tablet report, sending...
I20260928 14:50:07.359149 15355 ts_manager.cc:194] Registered new tserver with Master: 3a128f9f176b44fa9b761b8e4447e196 (127.14.241.193:45073)
I20260928 14:50:07.360714 15355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:59219
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:50:07.397395 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46591
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:07.873750 15550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:07.874035 15550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:07.874154 15550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:07.884332 15550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:07.884450 15550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:07.884609 15550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:07.896549 15550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46591
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:07.897977 15550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:07.899430 15550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:07.915077 15555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:07.915071 15556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:07.915091 15558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:07.916249 15550 server_base.cc:1034] running on GCE node
I20260928 14:50:07.916884 15550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:07.917846 15550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:07.919108 15550 hybrid_clock.cc:648] HybridClock initialized: now 1790607007919014 us; error 40 us; skew 500 ppm
I20260928 14:50:07.919479 15550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:07 dist-test-slave-vk40 krb5kdc[15310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607007, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:07.923740 15550 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:50:07.925752 15550 webserver.cc:460] Webserver started at http://127.14.241.194:38277/ using document root <none> and password file <none>
I20260928 14:50:07.926542 15550 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:07.926735 15550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:07.927070 15550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:07.930562 15550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "a0614e395a6e47d4a4b5fc3a1cdae1e9"
format_stamp: "Formatted at 2026-09-28 14:50:07 on dist-test-slave-vk40"
server_key: "7567a5561ab588fd566b53cc7ca8245b"
server_key_iv: "c9f1d1062f9364cf76b47413e3958a97"
server_key_version: "encryptionkey@0"
I20260928 14:50:07.931615 15550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "a0614e395a6e47d4a4b5fc3a1cdae1e9"
format_stamp: "Formatted at 2026-09-28 14:50:07 on dist-test-slave-vk40"
server_key: "7567a5561ab588fd566b53cc7ca8245b"
server_key_iv: "c9f1d1062f9364cf76b47413e3958a97"
server_key_version: "encryptionkey@0"
I20260928 14:50:07.938128 15550 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260928 14:50:07.942814 15565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:07.944332 15550 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260928 14:50:07.944509 15550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "a0614e395a6e47d4a4b5fc3a1cdae1e9"
format_stamp: "Formatted at 2026-09-28 14:50:07 on dist-test-slave-vk40"
server_key: "7567a5561ab588fd566b53cc7ca8245b"
server_key_iv: "c9f1d1062f9364cf76b47413e3958a97"
server_key_version: "encryptionkey@0"
I20260928 14:50:07.944717 15550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:07.965508 15550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:07.970137 15550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:07.970613 15550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:07.972375 15550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:07.974478 15550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:07.974586 15550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:07.974714 15550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:07.974776 15550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:08.034824 15550 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:35897
I20260928 14:50:08.034919 15677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:35897 every 8 connection(s)
I20260928 14:50:08.036178 15550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:08.045848 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 15550
I20260928 14:50:08.046187 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:50:08.046944 15303 external_mini_cluster.cc:1428] Setting key 5f4d8f7c309fa2d77c4179e656820e71
Sep 28 14:50:08 dist-test-slave-vk40 krb5kdc[15310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607007, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:08.063743 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:08.039111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:53317 (local address 127.14.241.254:46591)
0928 14:50:08.039905 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:08.039910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:08.041478 (+ 1568us) server_negotiation.cc:407] Connection header received
0928 14:50:08.044237 (+ 2759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:08.044242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:08.044358 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:08.044453 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:08.047230 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.048168 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.050907 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.051968 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.052087 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:08.057135 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:08.057142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:08.057145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:08.057181 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:08.060416 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.061391 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.061396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.061400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.061568 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.062098 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.062102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.062104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.062393 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:08.062511 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:08.063302 (+ 791us) server_negotiation.cc:299] Negotiation successful
0928 14:50:08.063486 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":571,"threads_started":1}
I20260928 14:50:08.064880 15681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:08.039817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46591 (local address 127.14.241.194:53317)
0928 14:50:08.041314 (+ 1497us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:08.041357 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:08.043957 (+ 2600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:08.044787 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:08.044796 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:08.045783 (+ 987us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:08.047026 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:08.047058 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.048362 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.048366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:08.050756 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:08.050770 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.052370 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.052379 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:08.052718 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:08.053590 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:08.053645 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:08.056861 (+ 3216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:08.060572 (+ 3711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:08.060583 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:08.060603 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:08.061230 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:08.061680 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:08.061685 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:08.061689 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:08.061955 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:08.062504 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:08.062513 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:08.062962 (+ 449us) client_negotiation.cc:769] Sending connection context
0928 14:50:08.063223 (+ 261us) client_negotiation.cc:240] Negotiation successful
0928 14:50:08.063658 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1177,"thread_start_us":818,"threads_started":1}
I20260928 14:50:08.066365 15678 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46591
I20260928 14:50:08.066676 15678 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:08.067602 15678 heartbeater.cc:511] Master 127.14.241.254:46591 requested a full tablet report, sending...
I20260928 14:50:08.069185 15355 ts_manager.cc:194] Registered new tserver with Master: a0614e395a6e47d4a4b5fc3a1cdae1e9 (127.14.241.194:35897)
I20260928 14:50:08.070225 15355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:53317
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:50:08.096510 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46591
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:50:08.363812 15544 heartbeater.cc:503] Master 127.14.241.254:46591 was elected leader, sending a full tablet report...
W20260928 14:50:08.490752 15684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:08.491031 15684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:08.491169 15684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:08.499418 15684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:08.499550 15684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:08.499713 15684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:08.512357 15684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46591
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:08.513900 15684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:08.515501 15684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:08.530701 15689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:08.530978 15690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:08.530990 15692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:08.532146 15684 server_base.cc:1034] running on GCE node
I20260928 14:50:08.532954 15684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:08.533964 15684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:08.535182 15684 hybrid_clock.cc:648] HybridClock initialized: now 1790607008535131 us; error 39 us; skew 500 ppm
I20260928 14:50:08.535580 15684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:08 dist-test-slave-vk40 krb5kdc[15310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607008, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:08.539912 15684 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:50:08.541980 15684 webserver.cc:460] Webserver started at http://127.14.241.195:37367/ using document root <none> and password file <none>
I20260928 14:50:08.542898 15684 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:08.543135 15684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:08.543619 15684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:08.548126 15684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "742ede1767ee43ce8b07e5974ee73ae0"
format_stamp: "Formatted at 2026-09-28 14:50:08 on dist-test-slave-vk40"
server_key: "8062106f8ce74c9c004a3525e971b2ee"
server_key_iv: "444dab4e6ea0b480527ca070e586c2cf"
server_key_version: "encryptionkey@0"
I20260928 14:50:08.549333 15684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "742ede1767ee43ce8b07e5974ee73ae0"
format_stamp: "Formatted at 2026-09-28 14:50:08 on dist-test-slave-vk40"
server_key: "8062106f8ce74c9c004a3525e971b2ee"
server_key_iv: "444dab4e6ea0b480527ca070e586c2cf"
server_key_version: "encryptionkey@0"
I20260928 14:50:08.556078 15684 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260928 14:50:08.560456 15699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:08.561789 15684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:08.561955 15684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "742ede1767ee43ce8b07e5974ee73ae0"
format_stamp: "Formatted at 2026-09-28 14:50:08 on dist-test-slave-vk40"
server_key: "8062106f8ce74c9c004a3525e971b2ee"
server_key_iv: "444dab4e6ea0b480527ca070e586c2cf"
server_key_version: "encryptionkey@0"
I20260928 14:50:08.562141 15684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:08.588620 15684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:08.594134 15684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:08.594529 15684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:08.595970 15684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:08.597995 15684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:08.598147 15684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:08.598337 15684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:08.598413 15684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:08.654278 15684 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:39887
I20260928 14:50:08.654322 15811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:39887 every 8 connection(s)
I20260928 14:50:08.655575 15684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:08.663638 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 15684
I20260928 14:50:08.663906 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:50:08.664657 15303 external_mini_cluster.cc:1428] Setting key aa483a45a6cd66b62a601f0fc35b98c4
Sep 28 14:50:08 dist-test-slave-vk40 krb5kdc[15310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607008, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:08.678833 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:08.658500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:39963 (local address 127.14.241.254:46591)
0928 14:50:08.658987 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:08.658993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:08.660838 (+ 1845us) server_negotiation.cc:407] Connection header received
0928 14:50:08.664208 (+ 3370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:08.664215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:08.664349 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:08.664463 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:08.666444 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.667351 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.669427 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.670007 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.670102 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:08.673701 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:08.673706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:08.673709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:08.673741 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:08.676237 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.676989 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.676992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.676994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.677101 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.677432 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.677437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.677439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.677677 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:08.677782 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:08.678426 (+ 644us) server_negotiation.cc:299] Negotiation successful
0928 14:50:08.678573 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":241,"threads_started":1}
I20260928 14:50:08.680154 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:08.659285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46591 (local address 127.14.241.195:39963)
0928 14:50:08.660650 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:08.660722 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:08.663897 (+ 3175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:08.664730 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:08.664743 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:08.665514 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:08.666291 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:08.666312 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.667507 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.667513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:08.669303 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:08.669316 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.670241 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.670247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:08.670514 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:08.671137 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:08.671159 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:08.673468 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:08.676395 (+ 2927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:08.676401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:08.676414 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:08.676887 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:08.677174 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:08.677177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:08.677179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:08.677356 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:08.677760 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:08.677765 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:08.678120 (+ 355us) client_negotiation.cc:769] Sending connection context
0928 14:50:08.678340 (+ 220us) client_negotiation.cc:240] Negotiation successful
0928 14:50:08.678765 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"thread_start_us":684,"threads_started":1}
I20260928 14:50:08.681798 15812 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46591
I20260928 14:50:08.682204 15812 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:08.683063 15812 heartbeater.cc:511] Master 127.14.241.254:46591 requested a full tablet report, sending...
I20260928 14:50:08.684665 15354 ts_manager.cc:194] Registered new tserver with Master: 742ede1767ee43ce8b07e5974ee73ae0 (127.14.241.195:39887)
I20260928 14:50:08.685930 15354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:39963
I20260928 14:50:08.694090 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:50:08 dist-test-slave-vk40 krb5kdc[15310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607008, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:50:08 dist-test-slave-vk40 krb5kdc[15310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607008, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:08.743070 15820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:08.727333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40052 (local address 127.14.241.193:45073)
0928 14:50:08.728028 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:08.728036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:08.728062 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:50:08.728127 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:08.728134 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:08.728376 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:08.728507 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:08.730487 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.731733 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.733109 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.733925 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.734017 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:08.737675 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:08.737687 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:08.737702 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:08.737738 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:08.740496 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.741095 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.741102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.741104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.741227 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.741728 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.741731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.741733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.742064 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:08.742294 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:08.742572 (+ 278us) server_negotiation.cc:299] Negotiation successful
0928 14:50:08.742766 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":402,"threads_started":1}
W20260928 14:50:08.744436 15459 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40052
Sep 28 14:50:08 dist-test-slave-vk40 krb5kdc[15310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:50:08 dist-test-slave-vk40 krb5kdc[15310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:08.786259 15820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:08.772422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40058 (local address 127.14.241.193:45073)
0928 14:50:08.772634 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:08.772641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:08.773059 (+ 418us) server_negotiation.cc:407] Connection header received
0928 14:50:08.773233 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:08.773237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:08.773300 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:08.773376 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:08.774576 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.775248 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.776437 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.776849 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.776948 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:08.780572 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:08.780578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:08.780581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:08.780616 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:08.783917 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.784580 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.784584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.784586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.784757 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.785126 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.785130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.785131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.785371 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:08.785540 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:08.785782 (+ 242us) server_negotiation.cc:299] Negotiation successful
0928 14:50:08.785959 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 14:50:08.787880 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 15416
I20260928 14:50:08.801278 15820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:08.788846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40064 (local address 127.14.241.193:45073)
0928 14:50:08.788997 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:08.789001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:08.789598 (+ 597us) server_negotiation.cc:407] Connection header received
0928 14:50:08.789723 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:08.789727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:08.789781 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:08.789848 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:08.791184 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.791862 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.793117 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.793652 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.793742 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:08.795686 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:08.795691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:08.795694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:08.795723 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:08.798918 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.799535 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.799539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.799541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.799682 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.800072 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.800076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.800078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.800425 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:08.800603 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:08.800832 (+ 229us) server_negotiation.cc:299] Negotiation successful
0928 14:50:08.800988 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:50:08.801954 15539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:08.884097 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 15416
I20260928 14:50:08.898074 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 15550
Sep 28 14:50:08 dist-test-slave-vk40 krb5kdc[15310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:50:08.913470 15827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:08.898961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49344 (local address 127.14.241.194:35897)
0928 14:50:08.899491 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:08.899498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:08.899523 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:50:08.899584 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:08.899590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:08.899795 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:08.899928 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:08.901073 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.902076 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.903343 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:08.903920 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:08.904025 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:08.907865 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:08.907879 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:08.907894 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:08.907934 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:08.910909 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.911534 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.911541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.911543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.911676 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:08.912089 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:08.912091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:08.912093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:08.912494 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:08.912723 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:08.912977 (+ 254us) server_negotiation.cc:299] Negotiation successful
0928 14:50:08.913177 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":254,"threads_started":1}
I20260928 14:50:08.914724 15673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:08.997850 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 15550
I20260928 14:50:09.015972 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 15684
Sep 28 14:50:09 dist-test-slave-vk40 krb5kdc[15310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:50:09.032243 15829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:09.016891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39074 (local address 127.14.241.195:39887)
0928 14:50:09.017634 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:09.017642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:09.017669 (+ 27us) server_negotiation.cc:407] Connection header received
0928 14:50:09.017730 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:09.017736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:09.017950 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:09.018095 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:09.019376 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:09.020605 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:09.021849 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:09.022410 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:09.022520 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:09.026611 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:09.026624 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:09.026638 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:09.026678 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:09.029442 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:09.030199 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:09.030207 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:09.030209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:09.030348 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:09.030734 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:09.030738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:09.030740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:09.031101 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:09.031346 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:09.031679 (+ 333us) server_negotiation.cc:299] Negotiation successful
0928 14:50:09.031896 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":406,"threads_started":1}
I20260928 14:50:09.033461 15807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:09.121313 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 15684
I20260928 14:50:09.135401 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 15323
I20260928 14:50:09.136580 15385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:09.239722 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 15323
2026-09-28T14:50:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4376 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:50:09 dist-test-slave-vk40 krb5kdc[15834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:50:09 dist-test-slave-vk40 krb5kdc[15834](info): set up 2 sockets
Sep 28 14:50:09 dist-test-slave-vk40 krb5kdc[15834](info): commencing operation
krb5kdc: starting...
W20260928 14:50:10.239161 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.004s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:50:10 dist-test-slave-vk40 krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:50:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:50:10.419760 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38791
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:38791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:10.734665 15847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:10.734984 15847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:10.735117 15847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:10.742463 15847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:10.742612 15847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:10.742691 15847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:10.742776 15847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:10.742837 15847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:10.752898 15847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:38791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:10.754280 15847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:10.755590 15847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:10.767829 15853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:10.767864 15855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:10.768332 15852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:10.769042 15847 server_base.cc:1034] running on GCE node
I20260928 14:50:10.770176 15847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:10.772219 15847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:10.773495 15847 hybrid_clock.cc:648] HybridClock initialized: now 1790607010773405 us; error 62 us; skew 500 ppm
I20260928 14:50:10.773934 15847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:10 dist-test-slave-vk40 krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607010, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:10.777143 15847 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:50:10.778735 15847 webserver.cc:460] Webserver started at http://127.14.241.254:38223/ using document root <none> and password file <none>
I20260928 14:50:10.779422 15847 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:10.779524 15847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:10.779801 15847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:10.782866 15847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "e5bbf7f1f4424788be38a20ae0de958e"
format_stamp: "Formatted at 2026-09-28 14:50:10 on dist-test-slave-vk40"
server_key: "071854405ebe46ecc574a3358b13af56"
server_key_iv: "98641077e67d9329b7a2df8042e73177"
server_key_version: "encryptionkey@0"
I20260928 14:50:10.783614 15847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "e5bbf7f1f4424788be38a20ae0de958e"
format_stamp: "Formatted at 2026-09-28 14:50:10 on dist-test-slave-vk40"
server_key: "071854405ebe46ecc574a3358b13af56"
server_key_iv: "98641077e67d9329b7a2df8042e73177"
server_key_version: "encryptionkey@0"
I20260928 14:50:10.788864 15847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260928 14:50:10.792338 15862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:10.793644 15847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:50:10.793828 15847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "e5bbf7f1f4424788be38a20ae0de958e"
format_stamp: "Formatted at 2026-09-28 14:50:10 on dist-test-slave-vk40"
server_key: "071854405ebe46ecc574a3358b13af56"
server_key_iv: "98641077e67d9329b7a2df8042e73177"
server_key_version: "encryptionkey@0"
I20260928 14:50:10.794003 15847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:10.820338 15847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:10.824645 15847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:10.824896 15847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:10.841801 15847 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:38791
I20260928 14:50:10.841811 15913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:38791 every 8 connection(s)
I20260928 14:50:10.843158 15847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:10.845187 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 15847
I20260928 14:50:10.845403 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:10.845974 15303 external_mini_cluster.cc:1428] Setting key 2d327e6a74946cc6ef5e891fa139857c
I20260928 14:50:10.847080 15914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 14:50:10 dist-test-slave-vk40 krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:10.860631 15914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e: Bootstrap starting.
I20260928 14:50:10.864727 15914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:10.865967 15914 log.cc:824] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:10.869386 15914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e: No bootstrap required, opened a new log
I20260928 14:50:10.869560 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:10.847887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43928 (local address 127.14.241.254:38791)
0928 14:50:10.848679 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:10.848694 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:10.848741 (+ 47us) server_negotiation.cc:407] Connection header received
0928 14:50:10.850498 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:10.850526 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:10.851050 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:10.851500 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:10.852827 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:10.854833 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:10.856090 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:10.857106 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:10.857287 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:10.861221 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:10.861231 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:10.861251 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:10.861303 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:10.865067 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:10.865745 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:10.865758 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:10.865767 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:10.865982 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:10.866323 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:10.866327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:10.866330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:10.866992 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:10.867205 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:10.867771 (+ 566us) server_negotiation.cc:299] Negotiation successful
0928 14:50:10.868217 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":327,"threads_started":1}
I20260928 14:50:10.877094 15914 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bbf7f1f4424788be38a20ae0de958e" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38791 } }
I20260928 14:50:10.877450 15914 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:10.877521 15914 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5bbf7f1f4424788be38a20ae0de958e, State: Initialized, Role: FOLLOWER
I20260928 14:50:10.878005 15914 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bbf7f1f4424788be38a20ae0de958e" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38791 } }
I20260928 14:50:10.878196 15914 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:10.878286 15914 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:10.878417 15914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:10.880481 15914 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bbf7f1f4424788be38a20ae0de958e" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38791 } }
I20260928 14:50:10.880937 15914 leader_election.cc:302] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5bbf7f1f4424788be38a20ae0de958e; no voters:
I20260928 14:50:10.881448 15914 leader_election.cc:288] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:10.881601 15919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:10.882182 15919 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [term 1 LEADER]: Becoming Leader. State: Replica: e5bbf7f1f4424788be38a20ae0de958e, State: Running, Role: LEADER
I20260928 14:50:10.882728 15919 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bbf7f1f4424788be38a20ae0de958e" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38791 } }
I20260928 14:50:10.885301 15914 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:10.885725 15921 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5bbf7f1f4424788be38a20ae0de958e. Latest consensus state: current_term: 1 leader_uuid: "e5bbf7f1f4424788be38a20ae0de958e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bbf7f1f4424788be38a20ae0de958e" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38791 } } }
I20260928 14:50:10.885906 15920 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5bbf7f1f4424788be38a20ae0de958e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bbf7f1f4424788be38a20ae0de958e" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38791 } } }
I20260928 14:50:10.886260 15921 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:10.886274 15920 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:10.890141 15923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:10.895535 15923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:10.905109 15923 catalog_manager.cc:1347] Generated new cluster ID: f133fa1070094a3db1d93a5b7f5c6796
I20260928 14:50:10.905238 15923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:10.920341 15923 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:10.921648 15923 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:10.940979 15923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5bbf7f1f4424788be38a20ae0de958e: Generated new TSK 0
I20260928 14:50:10.941840 15923 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:50:10.992882 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38791
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:11.316538 15940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:11.316891 15940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:11.317028 15940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:11.324236 15940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:11.324391 15940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:11.324546 15940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:11.334306 15940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38791
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:11.335613 15940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:11.336968 15940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:11.349232 15945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:11.349331 15946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:11.349371 15948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:11.350217 15940 server_base.cc:1034] running on GCE node
I20260928 14:50:11.350811 15940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:11.351742 15940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:11.352929 15940 hybrid_clock.cc:648] HybridClock initialized: now 1790607011352879 us; error 44 us; skew 500 ppm
I20260928 14:50:11.353216 15940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:11 dist-test-slave-vk40 krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607011, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:11.356418 15940 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:50:11.357911 15940 webserver.cc:460] Webserver started at http://127.14.241.193:46497/ using document root <none> and password file <none>
I20260928 14:50:11.358568 15940 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:11.358685 15940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:11.358989 15940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:11.361856 15940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "0ba73497033d4fcd8af97236c5b81f42"
format_stamp: "Formatted at 2026-09-28 14:50:11 on dist-test-slave-vk40"
server_key: "064f7623ef8d00d7c53def8426578bea"
server_key_iv: "bba50029dfcb8a0bd9f93e37baf5cc70"
server_key_version: "encryptionkey@0"
I20260928 14:50:11.362705 15940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "0ba73497033d4fcd8af97236c5b81f42"
format_stamp: "Formatted at 2026-09-28 14:50:11 on dist-test-slave-vk40"
server_key: "064f7623ef8d00d7c53def8426578bea"
server_key_iv: "bba50029dfcb8a0bd9f93e37baf5cc70"
server_key_version: "encryptionkey@0"
I20260928 14:50:11.367967 15940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260928 14:50:11.371526 15955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:11.372915 15940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260928 14:50:11.373064 15940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "0ba73497033d4fcd8af97236c5b81f42"
format_stamp: "Formatted at 2026-09-28 14:50:11 on dist-test-slave-vk40"
server_key: "064f7623ef8d00d7c53def8426578bea"
server_key_iv: "bba50029dfcb8a0bd9f93e37baf5cc70"
server_key_version: "encryptionkey@0"
I20260928 14:50:11.373260 15940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:11.386706 15940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:11.390862 15940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:11.391139 15940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:11.392052 15940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:11.393348 15940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:11.393421 15940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:11.393533 15940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:11.393591 15940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:11.422770 15940 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:37247
I20260928 14:50:11.422838 16067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:37247 every 8 connection(s)
I20260928 14:50:11.424017 15940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:11.430385 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 15940
I20260928 14:50:11.430576 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:50:11.431147 15303 external_mini_cluster.cc:1428] Setting key 2c655c09c5a72afdef17c5ae0c7da1c0
Sep 28 14:50:11 dist-test-slave-vk40 krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607011, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:11.443938 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:11.426493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:45481 (local address 127.14.241.254:38791)
0928 14:50:11.426939 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:11.426944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:11.427797 (+ 853us) server_negotiation.cc:407] Connection header received
0928 14:50:11.429661 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:11.429666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:11.429755 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:11.429850 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:11.431529 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:11.432422 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:11.434060 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:11.434467 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:11.434572 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:11.438150 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:11.438156 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:11.438159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:11.438191 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:11.440899 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:11.441608 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:11.441613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:11.441616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:11.441735 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:11.442142 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:11.442147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:11.442151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:11.442438 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:11.442580 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:11.443327 (+ 747us) server_negotiation.cc:299] Negotiation successful
0928 14:50:11.443558 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":246,"threads_started":1}
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
I20260928 14:50:11.445036 16071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:11.426967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38791 (local address 127.14.241.193:45481)
0928 14:50:11.427631 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:11.427672 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:11.429375 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:11.430119 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:11.430128 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:11.430724 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:11.431360 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:11.431381 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:11.432569 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:11.432573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:11.433923 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:11.433936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:11.434684 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:11.434692 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:11.434927 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:11.435570 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:11.435593 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:11.437912 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:11.441051 (+ 3139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:11.441061 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:11.441080 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:11.441491 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:11.441841 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:11.441846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:11.441850 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:11.442039 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:11.442562 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:11.442570 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:11.442969 (+ 399us) client_negotiation.cc:769] Sending connection context
0928 14:50:11.443218 (+ 249us) client_negotiation.cc:240] Negotiation successful
0928 14:50:11.443793 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":258,"threads_started":1}
I20260928 14:50:11.446597 16068 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38791
I20260928 14:50:11.446991 16068 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
I20260928 14:50:11.447882 16068 heartbeater.cc:511] Master 127.14.241.254:38791 requested a full tablet report, sending...
I20260928 14:50:11.450259 15879 ts_manager.cc:194] Registered new tserver with Master: 0ba73497033d4fcd8af97236c5b81f42 (127.14.241.193:37247)
I20260928 14:50:11.451689 15879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:45481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:50:11.468004 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38791
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:11.788456 16074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:11.788743 16074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:11.788838 16074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:11.796170 16074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:11.796314 16074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:11.796460 16074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:11.806181 16074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38791
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:11.807439 16074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:11.808861 16074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:11.821338 16079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:11.821338 16080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:11.821367 16082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:11.822148 16074 server_base.cc:1034] running on GCE node
I20260928 14:50:11.822721 16074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:11.823627 16074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:11.824815 16074 hybrid_clock.cc:648] HybridClock initialized: now 1790607011824752 us; error 58 us; skew 500 ppm
I20260928 14:50:11.825122 16074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:11 dist-test-slave-vk40 krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607011, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:11.832373 16074 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:50:11.833906 16074 webserver.cc:460] Webserver started at http://127.14.241.194:45509/ using document root <none> and password file <none>
I20260928 14:50:11.834570 16074 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:11.834668 16074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:11.834928 16074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:11.837821 16074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "8a36dc939ecf41e8bace08e6f82d5e73"
format_stamp: "Formatted at 2026-09-28 14:50:11 on dist-test-slave-vk40"
server_key: "100d223a6f8e93c4901ceaf2ae59b8c6"
server_key_iv: "012afb9644c2c0250a7f7d5ad1d5bbea"
server_key_version: "encryptionkey@0"
I20260928 14:50:11.838565 16074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "8a36dc939ecf41e8bace08e6f82d5e73"
format_stamp: "Formatted at 2026-09-28 14:50:11 on dist-test-slave-vk40"
server_key: "100d223a6f8e93c4901ceaf2ae59b8c6"
server_key_iv: "012afb9644c2c0250a7f7d5ad1d5bbea"
server_key_version: "encryptionkey@0"
I20260928 14:50:11.843796 16074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:50:11.847431 16089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:11.848923 16074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:11.849080 16074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "8a36dc939ecf41e8bace08e6f82d5e73"
format_stamp: "Formatted at 2026-09-28 14:50:11 on dist-test-slave-vk40"
server_key: "100d223a6f8e93c4901ceaf2ae59b8c6"
server_key_iv: "012afb9644c2c0250a7f7d5ad1d5bbea"
server_key_version: "encryptionkey@0"
I20260928 14:50:11.849263 16074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:11.875746 16074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:11.879879 16074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:11.880143 16074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:11.881101 16074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:11.882423 16074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:11.882498 16074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:11.882615 16074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:11.882666 16074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:11.913105 16074 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:39641
I20260928 14:50:11.913154 16201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:39641 every 8 connection(s)
I20260928 14:50:11.914347 16074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:11.923740 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 16074
I20260928 14:50:11.923930 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:50:11.924355 15303 external_mini_cluster.cc:1428] Setting key 3a27081045a4b9eeba36c0d8847392ec
Sep 28 14:50:11 dist-test-slave-vk40 krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607011, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:11.933406 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:11.916827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:50977 (local address 127.14.241.254:38791)
0928 14:50:11.916993 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:11.916998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:11.918181 (+ 1183us) server_negotiation.cc:407] Connection header received
0928 14:50:11.919969 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:11.919973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:11.920045 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:11.920133 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:11.921667 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:11.922404 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:11.923910 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:11.924383 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:11.924496 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:11.928084 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:11.928090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:11.928093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:11.928126 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:11.930508 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:11.931214 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:11.931218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:11.931220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:11.931344 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:11.931738 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:11.931743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:11.931745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:11.931995 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:11.932134 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:11.932859 (+ 725us) server_negotiation.cc:299] Negotiation successful
0928 14:50:11.933082 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 14:50:11.934407 16204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:11.917342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38791 (local address 127.14.241.194:50977)
0928 14:50:11.918009 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:11.918056 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:11.919705 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:11.920381 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:11.920390 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:11.920906 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:11.921500 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:11.921519 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:11.922524 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:11.922528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:11.923759 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:11.923774 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:11.924729 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:11.924734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:11.924938 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:11.925641 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:11.925663 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:11.927764 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:11.930639 (+ 2875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:11.930648 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:11.930666 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:11.931101 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:11.931442 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:11.931447 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:11.931451 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:11.931627 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:11.932113 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:11.932121 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:11.932504 (+ 383us) client_negotiation.cc:769] Sending connection context
0928 14:50:11.932751 (+ 247us) client_negotiation.cc:240] Negotiation successful
0928 14:50:11.933310 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":251,"threads_started":1}
I20260928 14:50:11.935783 16202 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38791
I20260928 14:50:11.936062 16202 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:11.936786 16202 heartbeater.cc:511] Master 127.14.241.254:38791 requested a full tablet report, sending...
I20260928 14:50:11.938261 15879 ts_manager.cc:194] Registered new tserver with Master: 8a36dc939ecf41e8bace08e6f82d5e73 (127.14.241.194:39641)
I20260928 14:50:11.939016 15879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:50977
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:50:11.963690 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38791
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:12.298055 16207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:12.298336 16207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:12.298439 16207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:12.305491 16207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:12.305593 16207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:12.305706 16207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:12.315086 16207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38791
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:12.316296 16207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:12.317566 16207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:12.330756 16213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:12.330789 16212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:12.330825 16215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:12.331411 16207 server_base.cc:1034] running on GCE node
I20260928 14:50:12.332010 16207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:12.332980 16207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:12.334194 16207 hybrid_clock.cc:648] HybridClock initialized: now 1790607012334130 us; error 49 us; skew 500 ppm
I20260928 14:50:12.334525 16207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:12 dist-test-slave-vk40 krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607012, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:12.337538 16207 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:50:12.338924 16207 webserver.cc:460] Webserver started at http://127.14.241.195:44733/ using document root <none> and password file <none>
I20260928 14:50:12.339581 16207 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:12.339684 16207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:12.340142 16207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:12.343883 16207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "9f093c4fd3fa41e697b0414217805742"
format_stamp: "Formatted at 2026-09-28 14:50:12 on dist-test-slave-vk40"
server_key: "2e72bcc990055ba6cb1c24dcd703a690"
server_key_iv: "2c34d77502efb62402e6e6a9c17a11cc"
server_key_version: "encryptionkey@0"
I20260928 14:50:12.344827 16207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "9f093c4fd3fa41e697b0414217805742"
format_stamp: "Formatted at 2026-09-28 14:50:12 on dist-test-slave-vk40"
server_key: "2e72bcc990055ba6cb1c24dcd703a690"
server_key_iv: "2c34d77502efb62402e6e6a9c17a11cc"
server_key_version: "encryptionkey@0"
I20260928 14:50:12.349921 16207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:50:12.353328 16222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:12.354768 16207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 14:50:12.354929 16207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "9f093c4fd3fa41e697b0414217805742"
format_stamp: "Formatted at 2026-09-28 14:50:12 on dist-test-slave-vk40"
server_key: "2e72bcc990055ba6cb1c24dcd703a690"
server_key_iv: "2c34d77502efb62402e6e6a9c17a11cc"
server_key_version: "encryptionkey@0"
I20260928 14:50:12.355113 16207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:12.391714 16207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:12.395641 16207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:12.395920 16207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:12.396920 16207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:12.398248 16207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:12.398324 16207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:12.398448 16207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:12.398499 16207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:12.427449 16207 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:46559
I20260928 14:50:12.427481 16334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:46559 every 8 connection(s)
I20260928 14:50:12.429036 16207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:12.429911 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 16207
I20260928 14:50:12.430096 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:50:12.430510 15303 external_mini_cluster.cc:1428] Setting key 045896e3ba2f718ce1360ef6fd298cba
Sep 28 14:50:12 dist-test-slave-vk40 krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607012, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:12.448504 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.431692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:33735 (local address 127.14.241.254:38791)
0928 14:50:12.431861 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.431866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.433218 (+ 1352us) server_negotiation.cc:407] Connection header received
0928 14:50:12.435133 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.435137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.435207 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.435290 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:12.436794 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.437536 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.439025 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.439430 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.439517 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.442767 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:12.442773 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:12.442776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:12.442812 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:12.445699 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.446489 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.446493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.446495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.446624 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.447020 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.447024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.447026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.447313 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:12.447427 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:12.448034 (+ 607us) server_negotiation.cc:299] Negotiation successful
0928 14:50:12.448206 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 14:50:12.449450 16337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.432246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38791 (local address 127.14.241.195:33735)
0928 14:50:12.433051 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:12.433096 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:12.434825 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:12.435492 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:12.435501 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:12.436027 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:12.436635 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.436653 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.437671 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.437675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.438894 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.438906 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.439588 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.439592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.439777 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.440405 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:12.440431 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:12.442534 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:12.445846 (+ 3312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:12.445855 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:12.445874 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:12.446350 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:12.446715 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:12.446718 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:12.446720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:12.446895 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:12.447406 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:12.447411 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:12.447731 (+ 320us) client_negotiation.cc:769] Sending connection context
0928 14:50:12.447958 (+ 227us) client_negotiation.cc:240] Negotiation successful
0928 14:50:12.448391 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":307,"threads_started":1}
I20260928 14:50:12.450819 16335 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38791
I20260928 14:50:12.451084 16335 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:12.451750 16335 heartbeater.cc:511] Master 127.14.241.254:38791 requested a full tablet report, sending...
I20260928 14:50:12.453125 15879 ts_manager.cc:194] Registered new tserver with Master: 9f093c4fd3fa41e697b0414217805742 (127.14.241.195:46559)
I20260928 14:50:12.453894 15879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:33735
I20260928 14:50:12.454651 16068 heartbeater.cc:503] Master 127.14.241.254:38791 was elected leader, sending a full tablet report...
I20260928 14:50:12.458017 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:50:12.475387 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.464008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.14.241.254:38791)
0928 14:50:12.464177 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.464182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.464665 (+ 483us) server_negotiation.cc:407] Connection header received
0928 14:50:12.464788 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.464791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.464857 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.464951 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:12.466333 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.467095 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.468245 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.468691 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.468782 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.470809 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:12.470813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:12.470817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:12.470851 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:12.473303 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.473880 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.473883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.473886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.474004 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.474272 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.474275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.474276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.474571 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:12.474754 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:12.474994 (+ 240us) server_negotiation.cc:299] Negotiation successful
0928 14:50:12.475150 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:50:12.481333 15879 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:50:12.488502 15879 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:12.522091 16348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.500099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37770 (local address 127.14.241.194:39641)
0928 14:50:12.513461 (+ 13362us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.513469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.513492 (+ 23us) server_negotiation.cc:407] Connection header received
0928 14:50:12.513584 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.513592 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.513865 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.514065 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:12.515331 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.516791 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.519078 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.520637 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.520823 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.521022 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:12.521583 (+ 561us) server_negotiation.cc:299] Negotiation successful
0928 14:50:12.521755 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13199,"thread_start_us":13072,"threads_started":1}
I20260928 14:50:12.522168 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.499927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:39641 (local address 127.0.0.1:37770)
0928 14:50:12.501077 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:12.501126 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:12.501300 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:12.514368 (+ 13068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:12.514373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:12.514492 (+ 119us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:12.515134 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.515143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.516941 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.516946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.518909 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.518922 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.520867 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.520874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.521316 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.521373 (+ 57us) client_negotiation.cc:769] Sending connection context
0928 14:50:12.521513 (+ 140us) client_negotiation.cc:240] Negotiation successful
0928 14:50:12.521734 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"mutex_wait_us":25,"spinlock_wait_cycles":133888,"thread_start_us":3666,"threads_started":1}
I20260928 14:50:12.523244 16352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.501697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34308 (local address 127.14.241.195:46559)
0928 14:50:12.510908 (+ 9211us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.510916 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.513476 (+ 2560us) server_negotiation.cc:407] Connection header received
0928 14:50:12.513584 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.513592 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.513867 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.514065 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:12.516459 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.517854 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.519810 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.521080 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.521288 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.521463 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:12.522770 (+ 1307us) server_negotiation.cc:299] Negotiation successful
0928 14:50:12.522945 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9036,"thread_start_us":8911,"threads_started":1}
I20260928 14:50:12.523283 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.500699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:46559 (local address 127.0.0.1:34308)
0928 14:50:12.513170 (+ 12471us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:12.513191 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:12.513341 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:12.514368 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:12.514373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:12.514503 (+ 130us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:12.515128 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.515142 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.517986 (+ 2844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.517991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.519621 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.519635 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.522415 (+ 2780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.522422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.522598 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.522631 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 14:50:12.522721 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 14:50:12.522856 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12335,"spinlock_wait_cycles":197888,"thread_start_us":12179,"threads_started":1}
I20260928 14:50:12.523348 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.500138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:37247 (local address 127.0.0.1:53316)
0928 14:50:12.510188 (+ 10050us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:12.510206 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:12.510344 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:12.514364 (+ 4020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:12.514371 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:12.514430 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:12.514817 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.514826 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.516927 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.516932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.520697 (+ 3765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.520712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.522496 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.522502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.522676 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.522708 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 14:50:12.522801 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 14:50:12.522932 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9923,"mutex_wait_us":1474,"spinlock_wait_cycles":6912,"thread_start_us":9787,"threads_started":1}
I20260928 14:50:12.524070 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.501358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.14.241.193:37247)
0928 14:50:12.512889 (+ 11531us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.512900 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.513536 (+ 636us) server_negotiation.cc:407] Connection header received
0928 14:50:12.513648 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.513655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.513919 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.514122 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:12.515004 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.516548 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.520901 (+ 4353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.522285 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.522487 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.522650 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:12.523601 (+ 951us) server_negotiation.cc:299] Negotiation successful
0928 14:50:12.523764 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11326,"thread_start_us":11167,"threads_started":1}
I20260928 14:50:12.526906 16137 tablet_service.cc:1467] Processing CreateTablet for tablet 4bfec0e6270940ee9e490bbf51190553 (DEFAULT_TABLE table=test-table [id=0eace8fc2f8d449096c4e58724ab0449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:12.527274 16270 tablet_service.cc:1467] Processing CreateTablet for tablet 4bfec0e6270940ee9e490bbf51190553 (DEFAULT_TABLE table=test-table [id=0eace8fc2f8d449096c4e58724ab0449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:12.527357 16003 tablet_service.cc:1467] Processing CreateTablet for tablet 4bfec0e6270940ee9e490bbf51190553 (DEFAULT_TABLE table=test-table [id=0eace8fc2f8d449096c4e58724ab0449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:12.528255 16137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bfec0e6270940ee9e490bbf51190553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:12.528635 16003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bfec0e6270940ee9e490bbf51190553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:12.528635 16270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bfec0e6270940ee9e490bbf51190553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:12.547916 16353 tablet_bootstrap.cc:492] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73: Bootstrap starting.
I20260928 14:50:12.549073 16354 tablet_bootstrap.cc:492] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742: Bootstrap starting.
I20260928 14:50:12.551007 16355 tablet_bootstrap.cc:492] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42: Bootstrap starting.
I20260928 14:50:12.552042 16353 tablet_bootstrap.cc:654] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:12.553474 16353 log.cc:824] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:12.555745 16354 tablet_bootstrap.cc:654] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:12.556071 16355 tablet_bootstrap.cc:654] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:12.557080 16354 log.cc:824] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:12.557673 16353 tablet_bootstrap.cc:492] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73: No bootstrap required, opened a new log
I20260928 14:50:12.557956 16353 ts_tablet_manager.cc:1397] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260928 14:50:12.557925 16355 log.cc:824] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:12.560050 16354 tablet_bootstrap.cc:492] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742: No bootstrap required, opened a new log
I20260928 14:50:12.560319 16354 ts_tablet_manager.cc:1397] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260928 14:50:12.560957 16355 tablet_bootstrap.cc:492] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42: No bootstrap required, opened a new log
I20260928 14:50:12.561421 16355 ts_tablet_manager.cc:1397] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260928 14:50:12.569242 16354 raft_consensus.cc:348] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a36dc939ecf41e8bace08e6f82d5e73" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39641 } } peers { permanent_uuid: "0ba73497033d4fcd8af97236c5b81f42" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 37247 } } peers { permanent_uuid: "9f093c4fd3fa41e697b0414217805742" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46559 } }
I20260928 14:50:12.569599 16354 raft_consensus.cc:374] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:12.569687 16354 raft_consensus.cc:729] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f093c4fd3fa41e697b0414217805742, State: Initialized, Role: FOLLOWER
I20260928 14:50:12.569624 16353 raft_consensus.cc:348] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a36dc939ecf41e8bace08e6f82d5e73" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39641 } } peers { permanent_uuid: "0ba73497033d4fcd8af97236c5b81f42" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 37247 } } peers { permanent_uuid: "9f093c4fd3fa41e697b0414217805742" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46559 } }
I20260928 14:50:12.570043 16353 raft_consensus.cc:374] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:12.570134 16353 raft_consensus.cc:729] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a36dc939ecf41e8bace08e6f82d5e73, State: Initialized, Role: FOLLOWER
I20260928 14:50:12.570293 16354 consensus_queue.cc:260] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a36dc939ecf41e8bace08e6f82d5e73" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39641 } } peers { permanent_uuid: "0ba73497033d4fcd8af97236c5b81f42" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 37247 } } peers { permanent_uuid: "9f093c4fd3fa41e697b0414217805742" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46559 } }
I20260928 14:50:12.570709 16353 consensus_queue.cc:260] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a36dc939ecf41e8bace08e6f82d5e73" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39641 } } peers { permanent_uuid: "0ba73497033d4fcd8af97236c5b81f42" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 37247 } } peers { permanent_uuid: "9f093c4fd3fa41e697b0414217805742" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46559 } }
I20260928 14:50:12.572201 16335 heartbeater.cc:503] Master 127.14.241.254:38791 was elected leader, sending a full tablet report...
I20260928 14:50:12.573179 16354 ts_tablet_manager.cc:1428] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260928 14:50:12.576561 16355 raft_consensus.cc:348] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a36dc939ecf41e8bace08e6f82d5e73" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39641 } } peers { permanent_uuid: "0ba73497033d4fcd8af97236c5b81f42" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 37247 } } peers { permanent_uuid: "9f093c4fd3fa41e697b0414217805742" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46559 } }
I20260928 14:50:12.577346 16355 raft_consensus.cc:374] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:12.577576 16355 raft_consensus.cc:729] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ba73497033d4fcd8af97236c5b81f42, State: Initialized, Role: FOLLOWER
I20260928 14:50:12.578467 16202 heartbeater.cc:503] Master 127.14.241.254:38791 was elected leader, sending a full tablet report...
I20260928 14:50:12.578413 16355 consensus_queue.cc:260] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a36dc939ecf41e8bace08e6f82d5e73" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39641 } } peers { permanent_uuid: "0ba73497033d4fcd8af97236c5b81f42" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 37247 } } peers { permanent_uuid: "9f093c4fd3fa41e697b0414217805742" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46559 } }
I20260928 14:50:12.579879 16353 ts_tablet_manager.cc:1428] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73: Time spent starting tablet: real 0.022s user 0.014s sys 0.007s
I20260928 14:50:12.581911 16355 ts_tablet_manager.cc:1428] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42: Time spent starting tablet: real 0.020s user 0.009s sys 0.011s
W20260928 14:50:12.666359 16203 tablet.cc:2369] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:50:12.676542 16069 tablet.cc:2369] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:50:12.680827 16336 tablet.cc:2369] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:50:12.699743 16359 raft_consensus.cc:482] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:12.700064 16359 raft_consensus.cc:504] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a36dc939ecf41e8bace08e6f82d5e73" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39641 } } peers { permanent_uuid: "0ba73497033d4fcd8af97236c5b81f42" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 37247 } } peers { permanent_uuid: "9f093c4fd3fa41e697b0414217805742" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46559 } }
I20260928 14:50:12.702081 16359 leader_election.cc:288] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a36dc939ecf41e8bace08e6f82d5e73 (127.14.241.194:39641), 0ba73497033d4fcd8af97236c5b81f42 (127.14.241.193:37247)
I20260928 14:50:12.709896 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.702531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:58991 (local address 127.14.241.193:37247)
0928 14:50:12.702723 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.702729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.703721 (+ 992us) server_negotiation.cc:407] Connection header received
0928 14:50:12.703864 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.703870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.703927 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.704014 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:12.704491 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.705390 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.708152 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.708985 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.709135 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.709210 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:12.709490 (+ 280us) server_negotiation.cc:299] Negotiation successful
0928 14:50:12.709610 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 14:50:12.709940 16337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.702496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:39641 (local address 127.14.241.195:36511)
0928 14:50:12.702892 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:12.702912 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:12.703062 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:12.703469 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:12.703475 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:12.703506 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:12.703877 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.703889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.705185 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.705189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.707689 (+ 2500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.707704 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.709213 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.709219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.709396 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.709419 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 14:50:12.709499 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 14:50:12.709634 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":31}
I20260928 14:50:12.710065 16348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.702468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:36511 (local address 127.14.241.194:39641)
0928 14:50:12.702672 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.702679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.703020 (+ 341us) server_negotiation.cc:407] Connection header received
0928 14:50:12.703234 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.703240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.703317 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.703422 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:12.704006 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.705015 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.707848 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.709066 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.709209 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.709277 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:12.709536 (+ 259us) server_negotiation.cc:299] Negotiation successful
0928 14:50:12.709660 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 14:50:12.710361 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.702559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:37247 (local address 127.14.241.195:58991)
0928 14:50:12.703616 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:12.703633 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:12.703757 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:12.704030 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:12.704033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:12.704053 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:12.704347 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.704358 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.705519 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.705524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.708017 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:12.708030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.709177 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.709183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:12.709341 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.709366 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 14:50:12.709446 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 14:50:12.709577 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":935,"mutex_wait_us":2,"thread_start_us":566,"threads_started":1}
I20260928 14:50:12.711086 16157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfec0e6270940ee9e490bbf51190553" candidate_uuid: "9f093c4fd3fa41e697b0414217805742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a36dc939ecf41e8bace08e6f82d5e73" is_pre_election: true
I20260928 14:50:12.711412 16023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfec0e6270940ee9e490bbf51190553" candidate_uuid: "9f093c4fd3fa41e697b0414217805742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ba73497033d4fcd8af97236c5b81f42" is_pre_election: true
I20260928 14:50:12.711603 16157 raft_consensus.cc:2454] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f093c4fd3fa41e697b0414217805742 in term 0.
I20260928 14:50:12.712092 16023 raft_consensus.cc:2454] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f093c4fd3fa41e697b0414217805742 in term 0.
I20260928 14:50:12.712505 16223 leader_election.cc:302] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a36dc939ecf41e8bace08e6f82d5e73, 9f093c4fd3fa41e697b0414217805742; no voters:
I20260928 14:50:12.713090 16359 raft_consensus.cc:2781] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:50:12.713374 16359 raft_consensus.cc:482] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:50:12.713557 16359 raft_consensus.cc:3037] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:12.717196 16359 raft_consensus.cc:504] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a36dc939ecf41e8bace08e6f82d5e73" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39641 } } peers { permanent_uuid: "0ba73497033d4fcd8af97236c5b81f42" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 37247 } } peers { permanent_uuid: "9f093c4fd3fa41e697b0414217805742" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46559 } }
I20260928 14:50:12.718130 16359 leader_election.cc:288] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [CANDIDATE]: Term 1 election: Requested vote from peers 8a36dc939ecf41e8bace08e6f82d5e73 (127.14.241.194:39641), 0ba73497033d4fcd8af97236c5b81f42 (127.14.241.193:37247)
I20260928 14:50:12.718560 16157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfec0e6270940ee9e490bbf51190553" candidate_uuid: "9f093c4fd3fa41e697b0414217805742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a36dc939ecf41e8bace08e6f82d5e73"
I20260928 14:50:12.718698 16023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfec0e6270940ee9e490bbf51190553" candidate_uuid: "9f093c4fd3fa41e697b0414217805742" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ba73497033d4fcd8af97236c5b81f42"
I20260928 14:50:12.718745 16157 raft_consensus.cc:3037] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:12.718995 16023 raft_consensus.cc:3037] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:12.721411 16157 raft_consensus.cc:2454] T 4bfec0e6270940ee9e490bbf51190553 P 8a36dc939ecf41e8bace08e6f82d5e73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f093c4fd3fa41e697b0414217805742 in term 1.
I20260928 14:50:12.721691 16023 raft_consensus.cc:2454] T 4bfec0e6270940ee9e490bbf51190553 P 0ba73497033d4fcd8af97236c5b81f42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f093c4fd3fa41e697b0414217805742 in term 1.
I20260928 14:50:12.721935 16223 leader_election.cc:302] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a36dc939ecf41e8bace08e6f82d5e73, 9f093c4fd3fa41e697b0414217805742; no voters:
I20260928 14:50:12.722321 16359 raft_consensus.cc:2781] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:12.723173 16359 raft_consensus.cc:686] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [term 1 LEADER]: Becoming Leader. State: Replica: 9f093c4fd3fa41e697b0414217805742, State: Running, Role: LEADER
I20260928 14:50:12.723788 16359 consensus_queue.cc:237] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a36dc939ecf41e8bace08e6f82d5e73" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39641 } } peers { permanent_uuid: "0ba73497033d4fcd8af97236c5b81f42" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 37247 } } peers { permanent_uuid: "9f093c4fd3fa41e697b0414217805742" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46559 } }
I20260928 14:50:12.729904 15878 catalog_manager.cc:5495] T 4bfec0e6270940ee9e490bbf51190553 P 9f093c4fd3fa41e697b0414217805742 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f093c4fd3fa41e697b0414217805742 (127.14.241.195). New cstate: current_term: 1 leader_uuid: "9f093c4fd3fa41e697b0414217805742" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a36dc939ecf41e8bace08e6f82d5e73" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ba73497033d4fcd8af97236c5b81f42" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f093c4fd3fa41e697b0414217805742" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46559 } health_report { overall_health: HEALTHY } } }
Sep 28 14:50:12 dist-test-slave-vk40 krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:12.775877 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.756605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.14.241.193:37247)
0928 14:50:12.756780 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.756785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.757520 (+ 735us) server_negotiation.cc:407] Connection header received
0928 14:50:12.757699 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.757703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.757769 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.757877 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:12.759607 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.760566 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.762077 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.762746 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.762853 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.768734 (+ 5881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:12.768756 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:12.768772 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:12.768820 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:12.772442 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.773305 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.773312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.773314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.773614 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.774037 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.774041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.774043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.774556 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:12.774882 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:12.775129 (+ 247us) server_negotiation.cc:299] Negotiation successful
0928 14:50:12.775297 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Sep 28 14:50:12 dist-test-slave-vk40 krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:50:12 dist-test-slave-vk40 krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:12.826315 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.807938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.14.241.193:37247)
0928 14:50:12.808127 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.808133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.808644 (+ 511us) server_negotiation.cc:407] Connection header received
0928 14:50:12.808818 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.808822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.808893 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.808978 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:12.810905 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.811980 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.813924 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.814633 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.814742 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.820046 (+ 5304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:12.820053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:12.820056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:12.820093 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:12.823796 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.824452 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.824458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.824462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.824589 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.824967 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.824972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.824974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.825237 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:12.825483 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:12.825744 (+ 261us) server_negotiation.cc:299] Negotiation successful
0928 14:50:12.825964 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Sep 28 14:50:12 dist-test-slave-vk40 krb5kdc[15834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:50:12 dist-test-slave-vk40 krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:12.873562 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.854969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.14.241.193:37247)
0928 14:50:12.855195 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.855202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.855537 (+ 335us) server_negotiation.cc:407] Connection header received
0928 14:50:12.855718 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.855722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.855785 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.855884 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:12.857713 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.858461 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.859993 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.860858 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.860969 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.866890 (+ 5921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:12.866897 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:12.866901 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:12.866937 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:12.870475 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.871275 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.871279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.871281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.871446 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.871771 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.871774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.871776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.872261 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:12.872580 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:12.872761 (+ 181us) server_negotiation.cc:299] Negotiation successful
0928 14:50:12.873031 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260928 14:50:12.875936 15983 tablet_service.cc:2908] Rejecting scan request for tablet 4bfec0e6270940ee9e490bbf51190553: Uninitialized: clean time has not yet been initialized
I20260928 14:50:12.879401 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 15940
I20260928 14:50:12.895278 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.880426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49666 (local address 127.14.241.193:37247)
0928 14:50:12.880631 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.880638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.881129 (+ 491us) server_negotiation.cc:407] Connection header received
0928 14:50:12.881249 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.881252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.881320 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.881397 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:12.883264 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.883943 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.885595 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:12.886024 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:12.886128 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:12.889438 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:12.889443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:12.889446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:12.889480 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:12.892338 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.893233 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.893239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.893243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.893390 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:12.893816 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:12.893820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:12.893823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:12.894187 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:12.894453 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:12.894728 (+ 275us) server_negotiation.cc:299] Negotiation successful
0928 14:50:12.894957 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 14:50:12.896130 16063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:12.985015 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 15940
I20260928 14:50:12.998687 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 16074
Sep 28 14:50:13 dist-test-slave-vk40 krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:50:13.012758 16348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:12.999551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.14.241.194:39641)
0928 14:50:12.999743 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:12.999748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:12.999764 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:50:12.999816 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:12.999820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:12.999889 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:12.999973 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:13.001171 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:13.001939 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:13.003177 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:13.003586 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:13.003695 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:13.007446 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:13.007460 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:13.007476 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:13.007514 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:13.010149 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:13.010765 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:13.010772 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:13.010774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:13.010905 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:13.011239 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:13.011243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:13.011245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:13.011633 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:13.011895 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:13.012197 (+ 302us) server_negotiation.cc:299] Negotiation successful
0928 14:50:13.012446 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:50:13.013482 16197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:13.099321 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 16074
I20260928 14:50:13.113030 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 16207
Sep 28 14:50:13 dist-test-slave-vk40 krb5kdc[15834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:50:13.130297 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:13.115436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48708 (local address 127.14.241.195:46559)
0928 14:50:13.115943 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:13.115949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:13.115968 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:50:13.116023 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:13.116026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:13.116092 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:13.116196 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:13.117688 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:13.118652 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:13.120203 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:13.120808 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:13.120923 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:13.124910 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:13.124925 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:13.124941 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:13.124976 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:13.127669 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:13.128327 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:13.128337 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:13.128341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:13.128507 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:13.128866 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:13.128868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:13.128870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:13.129189 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:13.129405 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:13.129697 (+ 292us) server_negotiation.cc:299] Negotiation successful
0928 14:50:13.129989 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":240,"threads_started":1}
I20260928 14:50:13.130966 16330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:13.219398 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 16207
I20260928 14:50:13.232563 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 15847
I20260928 14:50:13.233760 15909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:13.341888 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 15847
2026-09-28T14:50:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4103 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:50:13 dist-test-slave-vk40 krb5kdc[16384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:50:13 dist-test-slave-vk40 krb5kdc[16384](info): set up 2 sockets
Sep 28 14:50:13 dist-test-slave-vk40 krb5kdc[16384](info): commencing operation
krb5kdc: starting...
W20260928 14:50:14.335291 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:50:14 dist-test-slave-vk40 krb5kdc[16384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:50:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:50:14.507817 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41735
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:41735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:14.830940 16397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:14.831271 16397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:14.831394 16397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:14.838888 16397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:14.839018 16397 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 14:50:14.839084 16397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:14.839176 16397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:14.839237 16397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:14.839321 16397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:14.849350 16397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:41735
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:14.850666 16397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:14.851984 16397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:14.863181 16402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:14.863287 16405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:14.863852 16403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:14.864423 16397 server_base.cc:1034] running on GCE node
I20260928 14:50:14.865170 16397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:14.866608 16397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:14.867859 16397 hybrid_clock.cc:648] HybridClock initialized: now 1790607014867793 us; error 36 us; skew 500 ppm
I20260928 14:50:14.868186 16397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:14 dist-test-slave-vk40 krb5kdc[16384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607014, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:14.871338 16397 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:50:14.872891 16397 webserver.cc:460] Webserver started at http://127.14.241.254:34531/ using document root <none> and password file <none>
I20260928 14:50:14.873587 16397 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:14.873683 16397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:14.873996 16397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:14.877012 16397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "41a53cf2606d451f8414c922f4067e1a"
format_stamp: "Formatted at 2026-09-28 14:50:14 on dist-test-slave-vk40"
server_key: "7aa04df270424647a0e6e4a49ba47330"
server_key_iv: "5c481c5827803bd2bd0ff4422a305ac1"
server_key_version: "encryptionkey@0"
I20260928 14:50:14.877795 16397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "41a53cf2606d451f8414c922f4067e1a"
format_stamp: "Formatted at 2026-09-28 14:50:14 on dist-test-slave-vk40"
server_key: "7aa04df270424647a0e6e4a49ba47330"
server_key_iv: "5c481c5827803bd2bd0ff4422a305ac1"
server_key_version: "encryptionkey@0"
I20260928 14:50:14.883131 16397 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:50:14.886720 16412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:14.888134 16397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:50:14.888340 16397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "41a53cf2606d451f8414c922f4067e1a"
format_stamp: "Formatted at 2026-09-28 14:50:14 on dist-test-slave-vk40"
server_key: "7aa04df270424647a0e6e4a49ba47330"
server_key_iv: "5c481c5827803bd2bd0ff4422a305ac1"
server_key_version: "encryptionkey@0"
I20260928 14:50:14.888559 16397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:14.926050 16397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:14.930408 16397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:14.930666 16397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:14.950815 16397 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:41735
I20260928 14:50:14.950836 16473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:41735 every 8 connection(s)
I20260928 14:50:14.952212 16397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:14.954488 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 16397
I20260928 14:50:14.954684 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:14.955274 15303 external_mini_cluster.cc:1428] Setting key 508a67d85a686c6d8accce8eb18e591a
I20260928 14:50:14.956262 16474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 14:50:14 dist-test-slave-vk40 krb5kdc[16384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:14.970275 16474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a: Bootstrap starting.
I20260928 14:50:14.973626 16474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:14.974644 16474 log.cc:824] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:14.977497 16474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a: No bootstrap required, opened a new log
I20260928 14:50:14.977532 16477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:14.957216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56478 (local address 127.14.241.254:41735)
0928 14:50:14.957935 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:14.957947 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:14.957988 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:50:14.959611 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:14.959634 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:14.960062 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:14.960443 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:14.962073 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:14.963936 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:14.965475 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:14.966316 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:14.966489 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:14.970656 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:14.970664 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:14.970680 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:14.970726 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:14.973594 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:14.974283 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:14.974290 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:14.974297 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:14.974465 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:14.974828 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:14.974831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:14.974833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:14.975399 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:14.975710 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:14.976115 (+ 405us) server_negotiation.cc:299] Negotiation successful
0928 14:50:14.976548 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":304,"threads_started":1}
I20260928 14:50:14.985486 16474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a53cf2606d451f8414c922f4067e1a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41735 } }
I20260928 14:50:14.985872 16474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:14.985966 16474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a53cf2606d451f8414c922f4067e1a, State: Initialized, Role: FOLLOWER
I20260928 14:50:14.986615 16474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a53cf2606d451f8414c922f4067e1a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41735 } }
I20260928 14:50:14.986891 16474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:14.987021 16474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:14.987185 16474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:14.989379 16474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a53cf2606d451f8414c922f4067e1a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41735 } }
I20260928 14:50:14.989909 16474 leader_election.cc:302] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41a53cf2606d451f8414c922f4067e1a; no voters:
I20260928 14:50:14.990479 16474 leader_election.cc:288] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:14.990626 16479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:14.991276 16479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [term 1 LEADER]: Becoming Leader. State: Replica: 41a53cf2606d451f8414c922f4067e1a, State: Running, Role: LEADER
I20260928 14:50:14.991791 16479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a53cf2606d451f8414c922f4067e1a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41735 } }
I20260928 14:50:14.992607 16474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:14.994901 16480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41a53cf2606d451f8414c922f4067e1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a53cf2606d451f8414c922f4067e1a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41735 } } }
I20260928 14:50:14.994814 16481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41a53cf2606d451f8414c922f4067e1a. Latest consensus state: current_term: 1 leader_uuid: "41a53cf2606d451f8414c922f4067e1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a53cf2606d451f8414c922f4067e1a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41735 } } }
I20260928 14:50:14.995260 16481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:14.996373 16480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:14.996985 16486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:15.002313 16486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:15.011487 16486 catalog_manager.cc:1347] Generated new cluster ID: 74ffe2faff284eea907db628f61ee845
I20260928 14:50:15.011613 16486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:15.026149 16486 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:15.028127 16486 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:15.035696 16486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a: Generated new TSK 0
I20260928 14:50:15.036862 16486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:50:15.079756 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41735
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:15.411351 16500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:15.411649 16500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:15.411777 16500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:15.419303 16500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:15.419425 16500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:15.419553 16500 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:50:15.419660 16500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:15.429884 16500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.241.254:41735
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:15.431173 16500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:15.432516 16500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:15.445256 16506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:15.445314 16508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:15.445288 16505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:15.446015 16500 server_base.cc:1034] running on GCE node
I20260928 14:50:15.446542 16500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:15.447381 16500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:15.448585 16500 hybrid_clock.cc:648] HybridClock initialized: now 1790607015448514 us; error 39 us; skew 500 ppm
I20260928 14:50:15.448904 16500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:15 dist-test-slave-vk40 krb5kdc[16384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607015, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:15.451987 16500 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:50:15.453516 16500 webserver.cc:460] Webserver started at http://127.14.241.193:42551/ using document root <none> and password file <none>
I20260928 14:50:15.454190 16500 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:15.454283 16500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:15.454584 16500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:15.457595 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "06053ead850b45a2be9da92be62da7a5"
format_stamp: "Formatted at 2026-09-28 14:50:15 on dist-test-slave-vk40"
server_key: "38b135b7eb64cf59f8626da9ed9ec6e2"
server_key_iv: "f5a8e8ff8e0f182c399c750975f0a2b1"
server_key_version: "encryptionkey@0"
I20260928 14:50:15.458380 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "06053ead850b45a2be9da92be62da7a5"
format_stamp: "Formatted at 2026-09-28 14:50:15 on dist-test-slave-vk40"
server_key: "38b135b7eb64cf59f8626da9ed9ec6e2"
server_key_iv: "f5a8e8ff8e0f182c399c750975f0a2b1"
server_key_version: "encryptionkey@0"
I20260928 14:50:15.463577 16500 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 14:50:15.467126 16515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:15.468531 16500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:15.468696 16500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "06053ead850b45a2be9da92be62da7a5"
format_stamp: "Formatted at 2026-09-28 14:50:15 on dist-test-slave-vk40"
server_key: "38b135b7eb64cf59f8626da9ed9ec6e2"
server_key_iv: "f5a8e8ff8e0f182c399c750975f0a2b1"
server_key_version: "encryptionkey@0"
I20260928 14:50:15.468906 16500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:15.490504 16500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:15.494692 16500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:15.494952 16500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:15.497735 16500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:15.497824 16500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:15.497947 16500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:15.498054 16500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Sep 28 14:50:15 dist-test-slave-vk40 krb5kdc[16384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607015, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:15.525230 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:15.498197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:38223 (local address 127.14.241.254:41735)
0928 14:50:15.508652 (+ 10455us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:15.508660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:15.508688 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:50:15.508757 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:15.508763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:15.508854 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:15.508991 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:15.510620 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:15.511787 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:15.513907 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:15.514492 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:15.514638 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:15.519596 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:15.519602 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:15.519605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:15.519640 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:15.522135 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:15.522889 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:15.522893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:15.522895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:15.523014 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:15.523434 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:15.523438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:15.523440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:15.523722 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:15.523853 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:15.524633 (+ 780us) server_negotiation.cc:299] Negotiation successful
0928 14:50:15.524859 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10312,"thread_start_us":10205,"threads_started":1}
I20260928 14:50:15.526321 16525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:15.498963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41735 (local address 127.14.241.193:38223)
0928 14:50:15.499927 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:15.499977 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:15.502537 (+ 2560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:15.509279 (+ 6742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:15.509289 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:15.509836 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:15.510447 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:15.510465 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:15.511934 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:15.511937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:15.513674 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:15.513697 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:15.514783 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:15.514792 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:15.515178 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:15.516060 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:15.516104 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:15.519343 (+ 3239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:15.522284 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:15.522294 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:15.522313 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:15.522758 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:15.523116 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:15.523121 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:15.523125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:15.523314 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:15.523845 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:15.523853 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:15.524222 (+ 369us) client_negotiation.cc:769] Sending connection context
0928 14:50:15.524538 (+ 316us) client_negotiation.cc:240] Negotiation successful
0928 14:50:15.525136 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":365,"threads_started":1}
I20260928 14:50:15.545944 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:15.531945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41735 (local address 127.14.241.193:49253)
0928 14:50:15.532773 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:15.532794 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:15.532943 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:15.533236 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:15.533241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:15.533830 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:15.534222 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:15.534234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:15.535281 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:15.535286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:15.537071 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:15.537089 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:15.537996 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:15.538003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:15.538238 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:15.538838 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:15.538861 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:15.539955 (+ 1094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:15.543176 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:15.543181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:15.543189 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:15.543833 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:15.544249 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:15.544254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:15.544258 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:15.544464 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:15.545115 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:15.545119 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:15.545275 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 14:50:15.545380 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 14:50:15.545617 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":514,"threads_started":1}
I20260928 14:50:15.546139 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:15.531934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:49253 (local address 127.14.241.254:41735)
0928 14:50:15.532115 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:15.532122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:15.532885 (+ 763us) server_negotiation.cc:407] Connection header received
0928 14:50:15.533046 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:15.533049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:15.533112 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:15.533188 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:15.534343 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:15.535145 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:15.537228 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:15.537773 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:15.537900 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:15.540136 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:15.540140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:15.540143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:15.540174 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:15.542998 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:15.543960 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:15.543966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:15.543970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:15.544131 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:15.544609 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:15.544615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:15.544619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:15.545002 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:15.545165 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:15.545507 (+ 342us) server_negotiation.cc:299] Negotiation successful
0928 14:50:15.545767 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 14:50:15.557849 16500 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:42869
I20260928 14:50:15.557893 16635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:42869 every 8 connection(s)
I20260928 14:50:15.559041 16500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:15.560896 16636 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41735
I20260928 14:50:15.561082 16636 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:15.561655 16636 heartbeater.cc:511] Master 127.14.241.254:41735 requested a full tablet report, sending...
I20260928 14:50:15.563534 16429 ts_manager.cc:194] Registered new tserver with Master: 06053ead850b45a2be9da92be62da7a5 (127.14.241.193:42869)
I20260928 14:50:15.565034 16429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:38223
I20260928 14:50:15.568455 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 16500
I20260928 14:50:15.568629 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:50:15.569175 15303 external_mini_cluster.cc:1428] Setting key 129b1f9dc14ee573d2484783c7b4ecc8
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:50:15.607257 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41735
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:15.922896 16640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:15.923180 16640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:15.923274 16640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:15.930825 16640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:15.930974 16640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:15.931089 16640 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:50:15.931203 16640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:15.940925 16640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.241.254:41735
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:15.942176 16640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:15.943595 16640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:15.956666 16646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:15.956700 16648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:15.957261 16645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:15.957497 16640 server_base.cc:1034] running on GCE node
I20260928 14:50:15.958083 16640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:15.959040 16640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:15.960309 16640 hybrid_clock.cc:648] HybridClock initialized: now 1790607015960204 us; error 55 us; skew 500 ppm
I20260928 14:50:15.960634 16640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:15 dist-test-slave-vk40 krb5kdc[16384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607015, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:15.963840 16640 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:50:15.968079 16640 webserver.cc:460] Webserver started at http://127.14.241.194:38561/ using document root <none> and password file <none>
I20260928 14:50:15.969029 16640 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:15.969156 16640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:15.969470 16640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:15.972519 16640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "a07f10eedf79424d98d3c8544a88b00b"
format_stamp: "Formatted at 2026-09-28 14:50:15 on dist-test-slave-vk40"
server_key: "487e4f0e5ca4b4d9aece62289a008a19"
server_key_iv: "613b3f5eb664190276606e19a145adfc"
server_key_version: "encryptionkey@0"
I20260928 14:50:15.973299 16640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "a07f10eedf79424d98d3c8544a88b00b"
format_stamp: "Formatted at 2026-09-28 14:50:15 on dist-test-slave-vk40"
server_key: "487e4f0e5ca4b4d9aece62289a008a19"
server_key_iv: "613b3f5eb664190276606e19a145adfc"
server_key_version: "encryptionkey@0"
I20260928 14:50:15.978364 16640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:50:15.981835 16655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:15.983121 16640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:50:15.983259 16640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "a07f10eedf79424d98d3c8544a88b00b"
format_stamp: "Formatted at 2026-09-28 14:50:15 on dist-test-slave-vk40"
server_key: "487e4f0e5ca4b4d9aece62289a008a19"
server_key_iv: "613b3f5eb664190276606e19a145adfc"
server_key_version: "encryptionkey@0"
I20260928 14:50:15.983454 16640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:16.017221 16640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:16.021206 16640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:16.021456 16640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:16.024750 16640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:16.024852 16640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:16.024969 16640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:16.025034 16640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:50:16 dist-test-slave-vk40 krb5kdc[16384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607015, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:16.052678 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.026170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:45079 (local address 127.14.241.254:41735)
0928 14:50:16.026922 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:16.026928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:16.029025 (+ 2097us) server_negotiation.cc:407] Connection header received
0928 14:50:16.031541 (+ 2516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:16.031547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:16.031634 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:16.031722 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:16.034103 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.035210 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.037946 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.038614 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.038737 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.044473 (+ 5736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:16.044480 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:16.044485 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:16.044533 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:16.048110 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:16.049338 (+ 1228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:16.049348 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:16.049352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:16.049522 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:16.050322 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:16.050328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:16.050331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:16.050725 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:16.050898 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:16.052009 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0928 14:50:16.052300 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594}
I20260928 14:50:16.054630 16664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.025943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41735 (local address 127.14.241.194:45079)
0928 14:50:16.028824 (+ 2881us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:16.028893 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:16.031231 (+ 2338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:16.032145 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:16.032157 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:16.032884 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:16.033828 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.033874 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.035506 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.035513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.037713 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.037732 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.039020 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.039031 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.039379 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.040234 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:16.040273 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:16.044065 (+ 3792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:16.048387 (+ 4322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:16.048401 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:16.048423 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:16.049147 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:16.049755 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:16.049761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:16.049765 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:16.050183 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:16.050881 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:16.050893 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:16.051480 (+ 587us) client_negotiation.cc:769] Sending connection context
0928 14:50:16.051879 (+ 399us) client_negotiation.cc:240] Negotiation successful
0928 14:50:16.052608 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2589,"thread_start_us":2390,"threads_started":1}
I20260928 14:50:16.071688 16640 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:34981
I20260928 14:50:16.071764 16774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:34981 every 8 connection(s)
I20260928 14:50:16.073215 16640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:16.073575 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 16640
I20260928 14:50:16.073765 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:50:16.074280 15303 external_mini_cluster.cc:1428] Setting key 62546524768e9ef384e44802b02aa033
I20260928 14:50:16.075716 16775 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41735
I20260928 14:50:16.076263 16775 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:16.077077 16775 heartbeater.cc:511] Master 127.14.241.254:41735 requested a full tablet report, sending...
I20260928 14:50:16.079519 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.064152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:48687 (local address 127.14.241.254:41735)
0928 14:50:16.064416 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:16.064423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:16.065030 (+ 607us) server_negotiation.cc:407] Connection header received
0928 14:50:16.065231 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:16.065237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:16.065328 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:16.065436 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:16.066464 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.068053 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.069563 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.070189 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.070322 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.072566 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:16.072573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:16.072577 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:16.072620 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:16.076897 (+ 4277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:16.077624 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:16.077629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:16.077633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:16.077787 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:16.078137 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:16.078142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:16.078146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:16.078457 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:16.078591 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:16.078978 (+ 387us) server_negotiation.cc:299] Negotiation successful
0928 14:50:16.079185 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 14:50:16.079596 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.064215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41735 (local address 127.14.241.194:48687)
0928 14:50:16.064895 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:16.064918 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:16.065089 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:16.065482 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:16.065487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:16.066028 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:16.066306 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.066317 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.068245 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.068251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.069420 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.069434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.070372 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.070377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.070544 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.071299 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:16.071326 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:16.072350 (+ 1024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:16.077029 (+ 4679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:16.077034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:16.077039 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:16.077503 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:16.077899 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:16.077905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:16.077909 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:16.078032 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:16.078568 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:16.078573 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:16.078740 (+ 167us) client_negotiation.cc:769] Sending connection context
0928 14:50:16.078862 (+ 122us) client_negotiation.cc:240] Negotiation successful
0928 14:50:16.079183 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":387,"threads_started":1}
I20260928 14:50:16.082077 16428 ts_manager.cc:194] Registered new tserver with Master: a07f10eedf79424d98d3c8544a88b00b (127.14.241.194:34981)
I20260928 14:50:16.083051 16428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:45079
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:50:16.115976 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41735
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:16.437819 16779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:16.438114 16779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:16.438248 16779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:16.445429 16779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:16.445555 16779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:16.445676 16779 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:50:16.445794 16779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:16.455440 16779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.241.254:41735
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:16.456771 16779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:16.458127 16779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:16.470682 16784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:16.470683 16787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:16.470714 16785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:16.471469 16779 server_base.cc:1034] running on GCE node
I20260928 14:50:16.472083 16779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:16.473098 16779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:16.474334 16779 hybrid_clock.cc:648] HybridClock initialized: now 1790607016474262 us; error 46 us; skew 500 ppm
I20260928 14:50:16.474625 16779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:16 dist-test-slave-vk40 krb5kdc[16384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607016, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:16.477713 16779 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:50:16.479178 16779 webserver.cc:460] Webserver started at http://127.14.241.195:35789/ using document root <none> and password file <none>
I20260928 14:50:16.479869 16779 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:16.479967 16779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:16.480319 16779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:16.483343 16779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "d9a0445446d348efa3edea69e8eb2166"
format_stamp: "Formatted at 2026-09-28 14:50:16 on dist-test-slave-vk40"
server_key: "9257377574005a64bd40b9d41ad25b1b"
server_key_iv: "977733b964a4cebf0572f439b8331e43"
server_key_version: "encryptionkey@0"
I20260928 14:50:16.484078 16779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "d9a0445446d348efa3edea69e8eb2166"
format_stamp: "Formatted at 2026-09-28 14:50:16 on dist-test-slave-vk40"
server_key: "9257377574005a64bd40b9d41ad25b1b"
server_key_iv: "977733b964a4cebf0572f439b8331e43"
server_key_version: "encryptionkey@0"
I20260928 14:50:16.489158 16779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260928 14:50:16.492578 16794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:16.494042 16779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 14:50:16.494171 16779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "d9a0445446d348efa3edea69e8eb2166"
format_stamp: "Formatted at 2026-09-28 14:50:16 on dist-test-slave-vk40"
server_key: "9257377574005a64bd40b9d41ad25b1b"
server_key_iv: "977733b964a4cebf0572f439b8331e43"
server_key_version: "encryptionkey@0"
I20260928 14:50:16.494359 16779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:16.520783 16779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:16.525038 16779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:16.525283 16779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:16.527832 16779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:16.527938 16779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:16.528062 16779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:16.528169 16779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:50:16 dist-test-slave-vk40 krb5kdc[16384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607016, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:16.550807 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.528531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:34741 (local address 127.14.241.254:41735)
0928 14:50:16.528709 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:16.528715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:16.530286 (+ 1571us) server_negotiation.cc:407] Connection header received
0928 14:50:16.532459 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:16.532465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:16.532559 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:16.532644 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:16.534791 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.535784 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.538223 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.538719 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.538843 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.543886 (+ 5043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:16.543894 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:16.543899 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:16.543948 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:16.547260 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:16.548306 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:16.548313 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:16.548318 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:16.548500 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:16.548917 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:16.548921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:16.548925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:16.549306 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:16.549466 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:16.550129 (+ 663us) server_negotiation.cc:299] Negotiation successful
0928 14:50:16.550390 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:50:16.551656 16803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.529303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41735 (local address 127.14.241.195:34741)
0928 14:50:16.530130 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:16.530168 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:16.532139 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:16.532956 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:16.532969 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:16.533722 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:16.534598 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.534627 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.535987 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.535995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.538072 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.538088 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.539135 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.539142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.539468 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.540271 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:16.540335 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:16.543524 (+ 3189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:16.547421 (+ 3897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:16.547431 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:16.547455 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:16.548105 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:16.548613 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:16.548616 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:16.548619 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:16.548817 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:16.549413 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:16.549418 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:16.549752 (+ 334us) client_negotiation.cc:769] Sending connection context
0928 14:50:16.549972 (+ 220us) client_negotiation.cc:240] Negotiation successful
0928 14:50:16.550379 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":342,"threads_started":1}
I20260928 14:50:16.567770 16636 heartbeater.cc:503] Master 127.14.241.254:41735 was elected leader, sending a full tablet report...
I20260928 14:50:16.573611 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.559151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:38863 (local address 127.14.241.254:41735)
0928 14:50:16.559362 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:16.559369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:16.559804 (+ 435us) server_negotiation.cc:407] Connection header received
0928 14:50:16.559983 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:16.559986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:16.560050 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:16.560125 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:16.561330 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.562527 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.564025 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.564697 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.564831 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.566760 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:16.566767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:16.566772 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:16.566812 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:16.570221 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:16.571077 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:16.571083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:16.571086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:16.571257 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:16.571775 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:16.571781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:16.571785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:16.572157 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:16.572325 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:16.572979 (+ 654us) server_negotiation.cc:299] Negotiation successful
0928 14:50:16.573236 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:50:16.573815 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.559128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41735 (local address 127.14.241.195:38863)
0928 14:50:16.559689 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:16.559709 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:16.559867 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:16.560176 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:16.560181 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:16.560788 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:16.561146 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.561159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.562698 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.562704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.563884 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.563898 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.564923 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.564930 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.565065 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.565769 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:16.565792 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:16.566549 (+ 757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:16.570341 (+ 3792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:16.570347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:16.570355 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:16.570959 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:16.571407 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:16.571419 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:16.571423 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:16.571599 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:16.572426 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:16.572435 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:16.572663 (+ 228us) client_negotiation.cc:769] Sending connection context
0928 14:50:16.572855 (+ 192us) client_negotiation.cc:240] Negotiation successful
0928 14:50:16.573249 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":276,"threads_started":1}
I20260928 14:50:16.577097 16779 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:35515
I20260928 14:50:16.577148 16913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:35515 every 8 connection(s)
I20260928 14:50:16.578442 16779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:16.580396 16914 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41735
I20260928 14:50:16.580605 16914 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:16.581223 16914 heartbeater.cc:511] Master 127.14.241.254:41735 requested a full tablet report, sending...
I20260928 14:50:16.582161 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 16779
I20260928 14:50:16.582352 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:50:16.582650 16428 ts_manager.cc:194] Registered new tserver with Master: d9a0445446d348efa3edea69e8eb2166 (127.14.241.195:35515)
I20260928 14:50:16.582872 15303 external_mini_cluster.cc:1428] Setting key b87d1d5f5e2a704e976a93fe30f87131
I20260928 14:50:16.583711 16428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:34741
I20260928 14:50:16.585922 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:50:16 dist-test-slave-vk40 krb5kdc[16384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607016, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:50:16 dist-test-slave-vk40 krb5kdc[16384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607016, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:16.627254 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.614741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56480 (local address 127.14.241.254:41735)
0928 14:50:16.615005 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:16.615012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:16.615127 (+ 115us) server_negotiation.cc:407] Connection header received
0928 14:50:16.615295 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:16.615298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:16.615375 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:16.615452 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:16.616716 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.617435 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.618627 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.619038 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.619129 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.622691 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:16.622697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:16.622700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:16.622737 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:16.625219 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:16.625734 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:16.625737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:16.625740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:16.625845 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:16.626116 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:16.626118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:16.626120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:16.626387 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:16.626574 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:16.626760 (+ 186us) server_negotiation.cc:299] Negotiation successful
0928 14:50:16.626927 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260928 14:50:16.631122 16428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:50:16.638237 16428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:16.661520 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.649786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:34981 (local address 127.0.0.1:59760)
0928 14:50:16.651659 (+ 1873us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:16.651702 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:16.651911 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:16.652929 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:16.652938 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:16.652973 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:16.653430 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.653453 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.655496 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.655503 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.658151 (+ 2648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.658169 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.660443 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.660450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.660676 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.660730 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 14:50:16.660888 (+ 158us) client_negotiation.cc:240] Negotiation successful
0928 14:50:16.661097 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1590,"thread_start_us":1467,"threads_started":1}
I20260928 14:50:16.662379 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.649976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.14.241.194:34981)
0928 14:50:16.651238 (+ 1262us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:16.651249 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:16.652099 (+ 850us) server_negotiation.cc:407] Connection header received
0928 14:50:16.652206 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:16.652213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:16.652508 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:16.652685 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:16.653662 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.655326 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.658359 (+ 3033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.659901 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.660074 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.660271 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:16.661941 (+ 1670us) server_negotiation.cc:299] Negotiation successful
0928 14:50:16.662125 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1070,"thread_start_us":948,"threads_started":1}
I20260928 14:50:16.663139 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.650707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.14.241.193:42869)
0928 14:50:16.652255 (+ 1548us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:16.652265 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:16.652964 (+ 699us) server_negotiation.cc:407] Connection header received
0928 14:50:16.653077 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:16.653085 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:16.653409 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:16.653597 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:16.654587 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.656339 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.659237 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.660816 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.660989 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.661158 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:16.662704 (+ 1546us) server_negotiation.cc:299] Negotiation successful
0928 14:50:16.662874 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1371,"thread_start_us":1772,"threads_started":1}
I20260928 14:50:16.662555 16929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.650109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:42869 (local address 127.0.0.1:55046)
0928 14:50:16.652715 (+ 2606us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:16.652733 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:16.652850 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:16.653811 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:16.653816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:16.653841 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:16.654311 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.654324 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.656524 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.656530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.659043 (+ 2513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.659060 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.661701 (+ 2641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.661708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.661901 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.661938 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 14:50:16.662023 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:50:16.662169 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2477,"thread_start_us":2551,"threads_started":1}
I20260928 14:50:16.667025 16566 tablet_service.cc:1467] Processing CreateTablet for tablet 9b80965e84a246e9874bdbf2addcdb58 (DEFAULT_TABLE table=test-table [id=fd2b847c935d4cb4b06b2810a5231010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:16.666921 16705 tablet_service.cc:1467] Processing CreateTablet for tablet 9b80965e84a246e9874bdbf2addcdb58 (DEFAULT_TABLE table=test-table [id=fd2b847c935d4cb4b06b2810a5231010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:16.668392 16566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b80965e84a246e9874bdbf2addcdb58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:16.668386 16705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b80965e84a246e9874bdbf2addcdb58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:16.670163 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.651974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:35515 (local address 127.0.0.1:44764)
0928 14:50:16.663070 (+ 11096us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:16.663088 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:16.663212 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:16.664097 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:16.664101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:16.664125 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:16.664451 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.664462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.666290 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.666295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.668073 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.668086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.669401 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.669407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.669571 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.669604 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 14:50:16.669690 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 14:50:16.669820 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10969,"thread_start_us":10795,"threads_started":1}
I20260928 14:50:16.670173 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.652248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44764 (local address 127.14.241.195:35515)
0928 14:50:16.662440 (+ 10192us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:16.662449 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:16.663335 (+ 886us) server_negotiation.cc:407] Connection header received
0928 14:50:16.663448 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:16.663456 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:16.663695 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:16.663871 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:16.664841 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.666152 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.668221 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.669213 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.669402 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.669572 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:16.669738 (+ 166us) server_negotiation.cc:299] Negotiation successful
0928 14:50:16.669886 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9995,"thread_start_us":10173,"threads_started":1}
I20260928 14:50:16.673990 16844 tablet_service.cc:1467] Processing CreateTablet for tablet 9b80965e84a246e9874bdbf2addcdb58 (DEFAULT_TABLE table=test-table [id=fd2b847c935d4cb4b06b2810a5231010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:16.675452 16844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b80965e84a246e9874bdbf2addcdb58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:16.680998 16933 tablet_bootstrap.cc:492] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5: Bootstrap starting.
I20260928 14:50:16.684690 16933 tablet_bootstrap.cc:654] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:16.684981 16934 tablet_bootstrap.cc:492] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b: Bootstrap starting.
I20260928 14:50:16.686103 16933 log.cc:824] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:16.688221 16934 tablet_bootstrap.cc:654] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:16.689347 16933 tablet_bootstrap.cc:492] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5: No bootstrap required, opened a new log
I20260928 14:50:16.689481 16934 log.cc:824] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:16.689622 16933 ts_tablet_manager.cc:1397] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20260928 14:50:16.693122 16936 tablet_bootstrap.cc:492] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166: Bootstrap starting.
I20260928 14:50:16.696537 16936 tablet_bootstrap.cc:654] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:16.697455 16934 tablet_bootstrap.cc:492] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b: No bootstrap required, opened a new log
I20260928 14:50:16.697573 16936 log.cc:824] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:16.697734 16934 ts_tablet_manager.cc:1397] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260928 14:50:16.699131 16933 raft_consensus.cc:348] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.699586 16933 raft_consensus.cc:374] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:16.699683 16933 raft_consensus.cc:729] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06053ead850b45a2be9da92be62da7a5, State: Initialized, Role: FOLLOWER
I20260928 14:50:16.700337 16933 consensus_queue.cc:260] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.705538 16933 ts_tablet_manager.cc:1428] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5: Time spent starting tablet: real 0.016s user 0.009s sys 0.006s
I20260928 14:50:16.707685 16934 raft_consensus.cc:348] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.708072 16934 raft_consensus.cc:374] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:16.708173 16934 raft_consensus.cc:729] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07f10eedf79424d98d3c8544a88b00b, State: Initialized, Role: FOLLOWER
I20260928 14:50:16.708752 16936 tablet_bootstrap.cc:492] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166: No bootstrap required, opened a new log
I20260928 14:50:16.708824 16934 consensus_queue.cc:260] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.709046 16936 ts_tablet_manager.cc:1397] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.008s
I20260928 14:50:16.710656 16775 heartbeater.cc:503] Master 127.14.241.254:41735 was elected leader, sending a full tablet report...
I20260928 14:50:16.711383 16934 ts_tablet_manager.cc:1428] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260928 14:50:16.716928 16936 raft_consensus.cc:348] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.717257 16936 raft_consensus.cc:374] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:16.717329 16936 raft_consensus.cc:729] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a0445446d348efa3edea69e8eb2166, State: Initialized, Role: FOLLOWER
I20260928 14:50:16.717548 16939 raft_consensus.cc:482] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:16.717772 16939 raft_consensus.cc:504] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.717865 16936 consensus_queue.cc:260] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.718842 16914 heartbeater.cc:503] Master 127.14.241.254:41735 was elected leader, sending a full tablet report...
I20260928 14:50:16.719364 16936 ts_tablet_manager.cc:1428] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260928 14:50:16.719594 16939 leader_election.cc:288] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a07f10eedf79424d98d3c8544a88b00b (127.14.241.194:34981), d9a0445446d348efa3edea69e8eb2166 (127.14.241.195:35515)
I20260928 14:50:16.727710 16943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.719813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:34981 (local address 127.14.241.193:47805)
0928 14:50:16.721106 (+ 1293us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:16.721122 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:16.721288 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:16.721743 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:16.721747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:16.721782 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:16.722343 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.722356 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.723671 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.723676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.725959 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.725979 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.727056 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.727060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.727173 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.727197 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 14:50:16.727270 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 14:50:16.727364 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1185,"mutex_wait_us":59,"thread_start_us":1479,"threads_started":1}
I20260928 14:50:16.727738 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.720362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:47805 (local address 127.14.241.194:34981)
0928 14:50:16.720557 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:16.720562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:16.721195 (+ 633us) server_negotiation.cc:407] Connection header received
0928 14:50:16.721438 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:16.721443 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:16.721507 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:16.721560 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:16.722497 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.723513 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.726107 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.726860 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.726999 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.727052 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:16.727321 (+ 269us) server_negotiation.cc:299] Negotiation successful
0928 14:50:16.727462 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 14:50:16.728826 16725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b80965e84a246e9874bdbf2addcdb58" candidate_uuid: "06053ead850b45a2be9da92be62da7a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07f10eedf79424d98d3c8544a88b00b" is_pre_election: true
I20260928 14:50:16.729130 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.720213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:60791 (local address 127.14.241.195:35515)
0928 14:50:16.720402 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:16.720408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:16.721158 (+ 750us) server_negotiation.cc:407] Connection header received
0928 14:50:16.721333 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:16.721339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:16.721407 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:16.721478 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:16.722286 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.723361 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.727115 (+ 3754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.728184 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.728354 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.728405 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:16.728813 (+ 408us) server_negotiation.cc:299] Negotiation successful
0928 14:50:16.728922 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 14:50:16.729282 16942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.719801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:35515 (local address 127.14.241.193:60791)
0928 14:50:16.721041 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:16.721059 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:16.721209 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:16.721633 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:16.721637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:16.721661 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:16.722111 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.722123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.723523 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.723528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.725984 (+ 2456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:16.725993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.728454 (+ 2461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.728461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:16.728640 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.728673 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 14:50:16.728800 (+ 127us) client_negotiation.cc:240] Negotiation successful
0928 14:50:16.728923 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1106,"mutex_wait_us":52,"spinlock_wait_cycles":29440,"thread_start_us":905,"threads_started":1}
I20260928 14:50:16.729405 16725 raft_consensus.cc:2454] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06053ead850b45a2be9da92be62da7a5 in term 0.
I20260928 14:50:16.730016 16518 leader_election.cc:302] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06053ead850b45a2be9da92be62da7a5, a07f10eedf79424d98d3c8544a88b00b; no voters:
I20260928 14:50:16.730099 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b80965e84a246e9874bdbf2addcdb58" candidate_uuid: "06053ead850b45a2be9da92be62da7a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a0445446d348efa3edea69e8eb2166" is_pre_election: true
I20260928 14:50:16.730365 16939 raft_consensus.cc:2781] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:50:16.730513 16939 raft_consensus.cc:482] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:50:16.730620 16939 raft_consensus.cc:3037] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:16.730602 16864 raft_consensus.cc:2454] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06053ead850b45a2be9da92be62da7a5 in term 0.
I20260928 14:50:16.732972 16939 raft_consensus.cc:504] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.733495 16939 leader_election.cc:288] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [CANDIDATE]: Term 1 election: Requested vote from peers a07f10eedf79424d98d3c8544a88b00b (127.14.241.194:34981), d9a0445446d348efa3edea69e8eb2166 (127.14.241.195:35515)
I20260928 14:50:16.733940 16725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b80965e84a246e9874bdbf2addcdb58" candidate_uuid: "06053ead850b45a2be9da92be62da7a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07f10eedf79424d98d3c8544a88b00b"
I20260928 14:50:16.733994 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b80965e84a246e9874bdbf2addcdb58" candidate_uuid: "06053ead850b45a2be9da92be62da7a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a0445446d348efa3edea69e8eb2166"
I20260928 14:50:16.734150 16864 raft_consensus.cc:3037] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:16.734150 16725 raft_consensus.cc:3037] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:16.737212 16864 raft_consensus.cc:2454] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06053ead850b45a2be9da92be62da7a5 in term 1.
I20260928 14:50:16.737210 16725 raft_consensus.cc:2454] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06053ead850b45a2be9da92be62da7a5 in term 1.
I20260928 14:50:16.737617 16518 leader_election.cc:302] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 06053ead850b45a2be9da92be62da7a5, a07f10eedf79424d98d3c8544a88b00b, d9a0445446d348efa3edea69e8eb2166; no voters:
I20260928 14:50:16.737916 16939 raft_consensus.cc:2781] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:16.738353 16939 raft_consensus.cc:686] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 1 LEADER]: Becoming Leader. State: Replica: 06053ead850b45a2be9da92be62da7a5, State: Running, Role: LEADER
I20260928 14:50:16.738888 16939 consensus_queue.cc:237] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.743067 16429 catalog_manager.cc:5495] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06053ead850b45a2be9da92be62da7a5 (127.14.241.193). New cstate: current_term: 1 leader_uuid: "06053ead850b45a2be9da92be62da7a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } health_report { overall_health: UNKNOWN } } }
I20260928 14:50:16.775254 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:16.770125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55050 (local address 127.14.241.193:42869)
0928 14:50:16.770345 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:16.770350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:16.770408 (+ 58us) server_negotiation.cc:407] Connection header received
0928 14:50:16.770678 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:16.770682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:16.770765 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:16.770901 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:16.771180 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.771922 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.773299 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:16.773761 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:16.773858 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:16.774254 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:16.774752 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:16.774875 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:16.774937 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 14:50:16.775003 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 14:50:16.783329 16864 raft_consensus.cc:1261] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166 [term 1 FOLLOWER]: Refusing update from remote peer 06053ead850b45a2be9da92be62da7a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:16.783339 16725 raft_consensus.cc:1261] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b [term 1 FOLLOWER]: Refusing update from remote peer 06053ead850b45a2be9da92be62da7a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:16.784462 16939 consensus_queue.cc:1035] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:16.784771 16944 consensus_queue.cc:1035] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:16.797382 16947 mvcc.cc:205] Tried to move back new op lower bound from 7334326340734095360 to 7334326340565708800. Current Snapshot: MvccSnapshot[applied={T|T < 7334326340734095360}]
I20260928 14:50:16.797663 16948 mvcc.cc:205] Tried to move back new op lower bound from 7334326340734095360 to 7334326340565708800. Current Snapshot: MvccSnapshot[applied={T|T < 7334326340734095360}]
I20260928 14:50:16.817961 16428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:16.818267 16428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:16.822103 16428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a: Sending DeleteTablet for 3 replicas of tablet 9b80965e84a246e9874bdbf2addcdb58
I20260928 14:50:16.823324 16705 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b80965e84a246e9874bdbf2addcdb58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:16 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59760
I20260928 14:50:16.823684 16844 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b80965e84a246e9874bdbf2addcdb58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:16 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:44764
I20260928 14:50:16.824050 16566 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b80965e84a246e9874bdbf2addcdb58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:16 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:55046
I20260928 14:50:16.824980 16428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:50:16.825681 16428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:16.828609 16956 tablet_replica.cc:330] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b: stopping tablet replica
I20260928 14:50:16.829070 16956 raft_consensus.cc:2229] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:16.829586 16956 raft_consensus.cc:2258] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:16.830348 16957 tablet_replica.cc:330] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166: stopping tablet replica
I20260928 14:50:16.831027 16957 raft_consensus.cc:2229] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:16.831605 16957 raft_consensus.cc:2258] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:16.831916 16956 ts_tablet_manager.cc:1905] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:16.838611 16565 tablet_service.cc:1467] Processing CreateTablet for tablet 67f43ef5216846bd80e40e461a8bbead (DEFAULT_TABLE table=test-table [id=6b53194c19e541909d688b797e3da621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:16.839252 16565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67f43ef5216846bd80e40e461a8bbead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:16.839977 16956 ts_tablet_manager.cc:1918] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:50:16.840365 16956 log.cc:1196] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/9b80965e84a246e9874bdbf2addcdb58
I20260928 14:50:16.840617 16705 tablet_service.cc:1467] Processing CreateTablet for tablet 67f43ef5216846bd80e40e461a8bbead (DEFAULT_TABLE table=test-table [id=6b53194c19e541909d688b797e3da621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:16.841125 16705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67f43ef5216846bd80e40e461a8bbead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:16.841385 16956 ts_tablet_manager.cc:1939] T 9b80965e84a246e9874bdbf2addcdb58 P a07f10eedf79424d98d3c8544a88b00b: Deleting consensus metadata
I20260928 14:50:16.843631 16415 catalog_manager.cc:4841] TS a07f10eedf79424d98d3c8544a88b00b (127.14.241.194:34981): tablet 9b80965e84a246e9874bdbf2addcdb58 (table test-table [id=fd2b847c935d4cb4b06b2810a5231010]) successfully deleted
I20260928 14:50:16.845264 16844 tablet_service.cc:1467] Processing CreateTablet for tablet 67f43ef5216846bd80e40e461a8bbead (DEFAULT_TABLE table=test-table [id=6b53194c19e541909d688b797e3da621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:16.845860 16844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67f43ef5216846bd80e40e461a8bbead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:16.848448 16957 ts_tablet_manager.cc:1905] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:16.851516 16934 tablet_bootstrap.cc:492] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b: Bootstrap starting.
I20260928 14:50:16.853137 16934 tablet_bootstrap.cc:654] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:16.853575 16936 tablet_bootstrap.cc:492] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166: Bootstrap starting.
I20260928 14:50:16.855580 16936 tablet_bootstrap.cc:654] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:16.855738 16957 ts_tablet_manager.cc:1918] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:50:16.855913 16957 log.cc:1196] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/9b80965e84a246e9874bdbf2addcdb58
I20260928 14:50:16.846467 16933 tablet_bootstrap.cc:492] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5: Bootstrap starting.
I20260928 14:50:16.857601 16934 tablet_bootstrap.cc:492] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b: No bootstrap required, opened a new log
I20260928 14:50:16.857749 16934 ts_tablet_manager.cc:1397] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260928 14:50:16.858589 16934 raft_consensus.cc:348] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.858836 16934 raft_consensus.cc:374] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:16.858949 16934 raft_consensus.cc:729] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07f10eedf79424d98d3c8544a88b00b, State: Initialized, Role: FOLLOWER
I20260928 14:50:16.859265 16933 tablet_bootstrap.cc:654] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:16.859264 16934 consensus_queue.cc:260] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.860832 16957 ts_tablet_manager.cc:1939] T 9b80965e84a246e9874bdbf2addcdb58 P d9a0445446d348efa3edea69e8eb2166: Deleting consensus metadata
I20260928 14:50:16.862280 16934 ts_tablet_manager.cc:1428] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260928 14:50:16.862650 16414 catalog_manager.cc:4841] TS d9a0445446d348efa3edea69e8eb2166 (127.14.241.195:35515): tablet 9b80965e84a246e9874bdbf2addcdb58 (table test-table [id=fd2b847c935d4cb4b06b2810a5231010]) successfully deleted
I20260928 14:50:16.844897 16958 tablet_replica.cc:330] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5: stopping tablet replica
I20260928 14:50:16.865177 16958 raft_consensus.cc:2229] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:50:16.865953 16958 raft_consensus.cc:2258] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:16.868575 16958 ts_tablet_manager.cc:1905] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:16.872359 16933 tablet_bootstrap.cc:492] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5: No bootstrap required, opened a new log
I20260928 14:50:16.872543 16933 ts_tablet_manager.cc:1397] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.002s
I20260928 14:50:16.872615 16936 tablet_bootstrap.cc:492] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166: No bootstrap required, opened a new log
I20260928 14:50:16.872782 16936 ts_tablet_manager.cc:1397] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.005s
I20260928 14:50:16.873538 16933 raft_consensus.cc:348] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.873621 16936 raft_consensus.cc:348] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.873782 16933 raft_consensus.cc:374] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:16.873847 16936 raft_consensus.cc:374] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:16.873878 16933 raft_consensus.cc:729] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06053ead850b45a2be9da92be62da7a5, State: Initialized, Role: FOLLOWER
I20260928 14:50:16.873956 16936 raft_consensus.cc:729] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a0445446d348efa3edea69e8eb2166, State: Initialized, Role: FOLLOWER
I20260928 14:50:16.874197 16933 consensus_queue.cc:260] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.874235 16936 consensus_queue.cc:260] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:16.874938 16936 ts_tablet_manager.cc:1428] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260928 14:50:16.875957 16933 ts_tablet_manager.cc:1428] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:50:16.878404 16958 ts_tablet_manager.cc:1918] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:50:16.878647 16958 log.cc:1196] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/9b80965e84a246e9874bdbf2addcdb58
I20260928 14:50:16.879155 16958 ts_tablet_manager.cc:1939] T 9b80965e84a246e9874bdbf2addcdb58 P 06053ead850b45a2be9da92be62da7a5: Deleting consensus metadata
I20260928 14:50:16.880715 16416 catalog_manager.cc:4841] TS 06053ead850b45a2be9da92be62da7a5 (127.14.241.193:42869): tablet 9b80965e84a246e9874bdbf2addcdb58 (table test-table [id=fd2b847c935d4cb4b06b2810a5231010]) successfully deleted
W20260928 14:50:17.062273 16637 tablet.cc:2369] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:50:17.078593 16776 tablet.cc:2369] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:50:17.081027 16915 tablet.cc:2369] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:50:17.102626 16941 raft_consensus.cc:482] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:17.102973 16941 raft_consensus.cc:504] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.105492 16941 leader_election.cc:288] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06053ead850b45a2be9da92be62da7a5 (127.14.241.193:42869), a07f10eedf79424d98d3c8544a88b00b (127.14.241.194:34981)
I20260928 14:50:17.114538 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.105987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:42717 (local address 127.14.241.193:42869)
0928 14:50:17.106173 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.106178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.107167 (+ 989us) server_negotiation.cc:407] Connection header received
0928 14:50:17.107474 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.107481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.107650 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.107759 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:17.108314 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.109676 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.111762 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.113157 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.113322 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.113403 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.113976 (+ 573us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.114131 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:50:17.115466 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.105453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:42869 (local address 127.14.241.195:42717)
0928 14:50:17.107033 (+ 1580us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:17.107053 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:17.107236 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:17.107784 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:17.107789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:17.107815 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:17.108117 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.108131 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.109826 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.109832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.111550 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.111568 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.113563 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.113569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.113770 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.113805 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:50:17.114893 (+ 1088us) client_negotiation.cc:240] Negotiation successful
0928 14:50:17.115051 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1456,"thread_start_us":1312,"threads_started":1}
I20260928 14:50:17.116892 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.105996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:53949 (local address 127.14.241.194:34981)
0928 14:50:17.106171 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.106177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.107829 (+ 1652us) server_negotiation.cc:407] Connection header received
0928 14:50:17.108104 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.108110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.108185 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.108262 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:17.108804 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.110048 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.112744 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.114959 (+ 2215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.115129 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.115204 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.116252 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.116426 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 14:50:17.117229 16591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f43ef5216846bd80e40e461a8bbead" candidate_uuid: "d9a0445446d348efa3edea69e8eb2166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06053ead850b45a2be9da92be62da7a5" is_pre_election: true
I20260928 14:50:17.117949 16963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.105636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:34981 (local address 127.14.241.195:53949)
0928 14:50:17.107728 (+ 2092us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:17.107746 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:17.107941 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:17.108326 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:17.108330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:17.108352 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:17.108643 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.108653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.110187 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.110192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.112465 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.112482 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.115759 (+ 3277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.115769 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.116044 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.116079 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:50:17.117167 (+ 1088us) client_negotiation.cc:240] Negotiation successful
0928 14:50:17.117322 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1952,"thread_start_us":1736,"threads_started":1}
I20260928 14:50:17.118175 16591 raft_consensus.cc:2454] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9a0445446d348efa3edea69e8eb2166 in term 0.
I20260928 14:50:17.119058 16725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f43ef5216846bd80e40e461a8bbead" candidate_uuid: "d9a0445446d348efa3edea69e8eb2166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07f10eedf79424d98d3c8544a88b00b" is_pre_election: true
I20260928 14:50:17.119323 16725 raft_consensus.cc:2454] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9a0445446d348efa3edea69e8eb2166 in term 0.
I20260928 14:50:17.120067 16798 leader_election.cc:302] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06053ead850b45a2be9da92be62da7a5, d9a0445446d348efa3edea69e8eb2166; no voters:
I20260928 14:50:17.120635 16941 raft_consensus.cc:2781] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:50:17.120807 16941 raft_consensus.cc:482] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:50:17.120909 16941 raft_consensus.cc:3037] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:17.123276 16941 raft_consensus.cc:504] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.124183 16941 leader_election.cc:288] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [CANDIDATE]: Term 1 election: Requested vote from peers 06053ead850b45a2be9da92be62da7a5 (127.14.241.193:42869), a07f10eedf79424d98d3c8544a88b00b (127.14.241.194:34981)
I20260928 14:50:17.124696 16725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f43ef5216846bd80e40e461a8bbead" candidate_uuid: "d9a0445446d348efa3edea69e8eb2166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07f10eedf79424d98d3c8544a88b00b"
I20260928 14:50:17.124773 16591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f43ef5216846bd80e40e461a8bbead" candidate_uuid: "d9a0445446d348efa3edea69e8eb2166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06053ead850b45a2be9da92be62da7a5"
I20260928 14:50:17.124943 16725 raft_consensus.cc:3037] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:17.125039 16591 raft_consensus.cc:3037] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:17.128182 16591 raft_consensus.cc:2454] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9a0445446d348efa3edea69e8eb2166 in term 1.
I20260928 14:50:17.128567 16725 raft_consensus.cc:2454] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9a0445446d348efa3edea69e8eb2166 in term 1.
I20260928 14:50:17.128656 16798 leader_election.cc:302] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06053ead850b45a2be9da92be62da7a5, d9a0445446d348efa3edea69e8eb2166; no voters:
I20260928 14:50:17.129156 16941 raft_consensus.cc:2781] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:17.129773 16941 raft_consensus.cc:686] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 1 LEADER]: Becoming Leader. State: Replica: d9a0445446d348efa3edea69e8eb2166, State: Running, Role: LEADER
I20260928 14:50:17.130405 16941 consensus_queue.cc:237] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.134883 16428 catalog_manager.cc:5495] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9a0445446d348efa3edea69e8eb2166 (127.14.241.195). New cstate: current_term: 1 leader_uuid: "d9a0445446d348efa3edea69e8eb2166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } health_report { overall_health: HEALTHY } } }
Sep 28 14:50:17 dist-test-slave-vk40 krb5kdc[16384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607014, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:17.195192 16971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.170936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56488 (local address 127.14.241.254:41735)
0928 14:50:17.171443 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.171451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.171911 (+ 460us) server_negotiation.cc:407] Connection header received
0928 14:50:17.172131 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.172135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.172194 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.172320 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:17.174491 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.175239 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.177238 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.177654 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.177753 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.188688 (+ 10935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:17.188694 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:17.188698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:17.188734 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:17.191648 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.192731 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.192737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.192740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.192943 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.193385 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.193390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.193393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.193795 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:17.193924 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.194539 (+ 615us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.194802 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":368,"thread_start_us":254,"threads_started":1}
I20260928 14:50:17.195560 16972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.170944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41735 (local address 127.0.0.1:56488)
0928 14:50:17.171799 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:17.171820 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:17.171997 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:17.172384 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:17.172389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:17.173939 (+ 1550us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:17.174312 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.174327 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.175398 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.175404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.177092 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.177109 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.177832 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.177838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.178014 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.184879 (+ 6865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:17.184910 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:17.188460 (+ 3550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:17.191797 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:17.191813 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:17.191822 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:17.192607 (+ 785us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:17.193040 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:17.193045 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:17.193049 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:17.193275 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:17.193916 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:17.193945 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:17.194174 (+ 229us) client_negotiation.cc:769] Sending connection context
0928 14:50:17.194410 (+ 236us) client_negotiation.cc:240] Negotiation successful
0928 14:50:17.194937 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":192,"thread_start_us":549,"threads_started":1}
I20260928 14:50:17.202414 16428 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:56488:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 14:50:17.203655 16428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:17.211932 16705 tablet_service.cc:1467] Processing CreateTablet for tablet c45fbbaacc9e464d994fc3273762c3b2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aefdc622866c42619d8c5728a226ce29]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:50:17.211843 16565 tablet_service.cc:1467] Processing CreateTablet for tablet c45fbbaacc9e464d994fc3273762c3b2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aefdc622866c42619d8c5728a226ce29]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:50:17.211953 16844 tablet_service.cc:1467] Processing CreateTablet for tablet c45fbbaacc9e464d994fc3273762c3b2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aefdc622866c42619d8c5728a226ce29]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:50:17.212529 16705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c45fbbaacc9e464d994fc3273762c3b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:17.212571 16844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c45fbbaacc9e464d994fc3273762c3b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:17.212603 16565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c45fbbaacc9e464d994fc3273762c3b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:17.220798 16936 tablet_bootstrap.cc:492] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166: Bootstrap starting.
I20260928 14:50:17.221177 16934 tablet_bootstrap.cc:492] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: Bootstrap starting.
I20260928 14:50:17.222790 16933 tablet_bootstrap.cc:492] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5: Bootstrap starting.
I20260928 14:50:17.223985 16934 tablet_bootstrap.cc:654] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:17.224171 16936 tablet_bootstrap.cc:654] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:17.224993 16933 tablet_bootstrap.cc:654] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:17.233037 16933 tablet_bootstrap.cc:492] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5: No bootstrap required, opened a new log
I20260928 14:50:17.233230 16933 ts_tablet_manager.cc:1397] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260928 14:50:17.233796 16936 tablet_bootstrap.cc:492] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166: No bootstrap required, opened a new log
I20260928 14:50:17.234102 16936 ts_tablet_manager.cc:1397] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260928 14:50:17.234575 16933 raft_consensus.cc:348] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.235280 16934 tablet_bootstrap.cc:492] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: No bootstrap required, opened a new log
I20260928 14:50:17.235255 16936 raft_consensus.cc:348] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.235427 16933 raft_consensus.cc:374] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:17.235482 16934 ts_tablet_manager.cc:1397] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.009s
I20260928 14:50:17.235600 16933 raft_consensus.cc:729] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06053ead850b45a2be9da92be62da7a5, State: Initialized, Role: FOLLOWER
I20260928 14:50:17.235644 16936 raft_consensus.cc:374] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:17.235803 16936 raft_consensus.cc:729] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a0445446d348efa3edea69e8eb2166, State: Initialized, Role: FOLLOWER
I20260928 14:50:17.236033 16933 consensus_queue.cc:260] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.236200 16936 consensus_queue.cc:260] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.236311 16934 raft_consensus.cc:348] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.236534 16934 raft_consensus.cc:374] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:17.236611 16934 raft_consensus.cc:729] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07f10eedf79424d98d3c8544a88b00b, State: Initialized, Role: FOLLOWER
I20260928 14:50:17.236672 16944 tablet_replica.cc:439] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } } }
I20260928 14:50:17.236903 16933 ts_tablet_manager.cc:1428] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260928 14:50:17.236970 16944 tablet_replica.cc:442] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:17.236840 16934 consensus_queue.cc:260] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.237435 16934 ts_tablet_manager.cc:1428] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:50:17.237488 16940 tablet_replica.cc:439] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } } }
I20260928 14:50:17.237967 16941 tablet_replica.cc:439] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } } }
I20260928 14:50:17.238222 16941 tablet_replica.cc:442] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:17.238263 16940 tablet_replica.cc:442] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:17.239230 16936 ts_tablet_manager.cc:1428] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260928 14:50:17.321460 16940 raft_consensus.cc:482] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:17.321709 16940 raft_consensus.cc:504] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.323493 16940 leader_election.cc:288] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06053ead850b45a2be9da92be62da7a5 (127.14.241.193:42869), d9a0445446d348efa3edea69e8eb2166 (127.14.241.195:35515)
I20260928 14:50:17.332392 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.323942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:49607 (local address 127.14.241.193:42869)
0928 14:50:17.324129 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.324135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.324742 (+ 607us) server_negotiation.cc:407] Connection header received
0928 14:50:17.325082 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.325089 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.325170 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.325232 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:17.326333 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.327534 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.329575 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.331015 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.331230 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.331313 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.331625 (+ 312us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.331837 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 14:50:17.332880 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.324130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:35515 (local address 127.14.241.194:55225)
0928 14:50:17.325748 (+ 1618us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:17.325766 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:17.325911 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:17.326639 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:17.326643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:17.326669 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:17.327316 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.327329 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.328829 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.328834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.330596 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.330608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.332056 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.332062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.332222 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.332251 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 14:50:17.332363 (+ 112us) client_negotiation.cc:240] Negotiation successful
0928 14:50:17.332495 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1480,"spinlock_wait_cycles":768,"thread_start_us":1306,"threads_started":1}
I20260928 14:50:17.333419 16977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.323884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:42869 (local address 127.14.241.194:49607)
0928 14:50:17.324585 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:17.324640 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:17.324871 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:17.325489 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:17.325493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:17.325563 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:17.326104 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.326118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.327639 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.327645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.329345 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.329363 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.331241 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.331247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.331428 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.331464 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 14:50:17.332666 (+ 1202us) client_negotiation.cc:240] Negotiation successful
0928 14:50:17.332798 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"spinlock_wait_cycles":47616,"thread_start_us":489,"threads_started":1}
I20260928 14:50:17.333484 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.323852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:55225 (local address 127.14.241.195:35515)
0928 14:50:17.326324 (+ 2472us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.326330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.326349 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:50:17.326426 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.326432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.326502 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.326597 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:17.327480 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.328676 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.330747 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.331696 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.331851 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.331915 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.333093 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.333235 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2351,"thread_start_us":2252,"threads_started":1}
I20260928 14:50:17.333968 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45fbbaacc9e464d994fc3273762c3b2" candidate_uuid: "a07f10eedf79424d98d3c8544a88b00b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a0445446d348efa3edea69e8eb2166" is_pre_election: true
I20260928 14:50:17.334225 16864 raft_consensus.cc:2454] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07f10eedf79424d98d3c8544a88b00b in term 0.
I20260928 14:50:17.334157 16591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45fbbaacc9e464d994fc3273762c3b2" candidate_uuid: "a07f10eedf79424d98d3c8544a88b00b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06053ead850b45a2be9da92be62da7a5" is_pre_election: true
I20260928 14:50:17.334408 16591 raft_consensus.cc:2454] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07f10eedf79424d98d3c8544a88b00b in term 0.
I20260928 14:50:17.334692 16657 leader_election.cc:302] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a07f10eedf79424d98d3c8544a88b00b, d9a0445446d348efa3edea69e8eb2166; no voters:
I20260928 14:50:17.335086 16940 raft_consensus.cc:2781] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:50:17.335204 16940 raft_consensus.cc:482] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:50:17.335301 16940 raft_consensus.cc:3037] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:17.337425 16940 raft_consensus.cc:504] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.337951 16940 leader_election.cc:288] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [CANDIDATE]: Term 1 election: Requested vote from peers 06053ead850b45a2be9da92be62da7a5 (127.14.241.193:42869), d9a0445446d348efa3edea69e8eb2166 (127.14.241.195:35515)
I20260928 14:50:17.338469 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45fbbaacc9e464d994fc3273762c3b2" candidate_uuid: "a07f10eedf79424d98d3c8544a88b00b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a0445446d348efa3edea69e8eb2166"
I20260928 14:50:17.338472 16591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c45fbbaacc9e464d994fc3273762c3b2" candidate_uuid: "a07f10eedf79424d98d3c8544a88b00b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06053ead850b45a2be9da92be62da7a5"
I20260928 14:50:17.338675 16864 raft_consensus.cc:3037] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:17.338708 16591 raft_consensus.cc:3037] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:17.341442 16864 raft_consensus.cc:2454] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07f10eedf79424d98d3c8544a88b00b in term 1.
I20260928 14:50:17.341853 16657 leader_election.cc:302] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a07f10eedf79424d98d3c8544a88b00b, d9a0445446d348efa3edea69e8eb2166; no voters:
I20260928 14:50:17.342046 16591 raft_consensus.cc:2454] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07f10eedf79424d98d3c8544a88b00b in term 1.
I20260928 14:50:17.342156 16940 raft_consensus.cc:2781] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:17.342669 16940 raft_consensus.cc:686] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [term 1 LEADER]: Becoming Leader. State: Replica: a07f10eedf79424d98d3c8544a88b00b, State: Running, Role: LEADER
I20260928 14:50:17.343096 16940 consensus_queue.cc:237] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } }
I20260928 14:50:17.345009 16979 tablet_replica.cc:439] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: TxnStatusTablet state changed. Reason: New leader a07f10eedf79424d98d3c8544a88b00b. Latest consensus state: current_term: 1 leader_uuid: "a07f10eedf79424d98d3c8544a88b00b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } } }
I20260928 14:50:17.345499 16979 tablet_replica.cc:442] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:50:17.346990 16981 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 14:50:17.346900 16429 catalog_manager.cc:5495] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b reported cstate change: term changed from 0 to 1, leader changed from <none> to a07f10eedf79424d98d3c8544a88b00b (127.14.241.194). New cstate: current_term: 1 leader_uuid: "a07f10eedf79424d98d3c8544a88b00b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } health_report { overall_health: UNKNOWN } } }
I20260928 14:50:17.347303 16981 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 14:50:17.392818 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.387226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59772 (local address 127.14.241.194:34981)
0928 14:50:17.387410 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.387416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.387520 (+ 104us) server_negotiation.cc:407] Connection header received
0928 14:50:17.387810 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.387814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.387920 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.388063 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:17.388509 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.389275 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.390647 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.391093 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.391200 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.391673 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:17.392219 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:17.392381 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.392515 (+ 134us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.392583 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:50:17.393040 16972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.387212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:34981 (local address 127.0.0.1:59772)
0928 14:50:17.387434 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:17.387457 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:17.387611 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:17.388082 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:17.388087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:17.388098 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:50:17.388376 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.388390 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.389414 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.389420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.390510 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.390522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.391243 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.391250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.391433 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.391488 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:50:17.392364 (+ 876us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:50:17.392391 (+ 27us) client_negotiation.cc:769] Sending connection context
0928 14:50:17.392490 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 14:50:17.392661 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260928 14:50:17.399425 16864 raft_consensus.cc:1261] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166 [term 1 FOLLOWER]: Refusing update from remote peer a07f10eedf79424d98d3c8544a88b00b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:17.399730 16591 raft_consensus.cc:1261] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5 [term 1 FOLLOWER]: Refusing update from remote peer a07f10eedf79424d98d3c8544a88b00b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:17.400173 16940 consensus_queue.cc:1035] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:17.400549 16979 consensus_queue.cc:1035] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:17.404546 16941 tablet_replica.cc:439] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166: TxnStatusTablet state changed. Reason: New leader a07f10eedf79424d98d3c8544a88b00b. Latest consensus state: current_term: 1 leader_uuid: "a07f10eedf79424d98d3c8544a88b00b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } } }
I20260928 14:50:17.404911 16941 tablet_replica.cc:442] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:17.406966 16985 mvcc.cc:205] Tried to move back new op lower bound from 7334326343259664384 to 7334326343041216512. Current Snapshot: MvccSnapshot[applied={T|T < 7334326343259664384}]
I20260928 14:50:17.412396 16940 tablet_replica.cc:439] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a07f10eedf79424d98d3c8544a88b00b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } } }
I20260928 14:50:17.412833 16940 tablet_replica.cc:442] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:50:17.415429 16941 tablet_replica.cc:439] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a07f10eedf79424d98d3c8544a88b00b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } } }
I20260928 14:50:17.415745 16941 tablet_replica.cc:442] T c45fbbaacc9e464d994fc3273762c3b2 P d9a0445446d348efa3edea69e8eb2166: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:17.421173 16944 tablet_replica.cc:439] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5: TxnStatusTablet state changed. Reason: New leader a07f10eedf79424d98d3c8544a88b00b. Latest consensus state: current_term: 1 leader_uuid: "a07f10eedf79424d98d3c8544a88b00b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } } }
I20260928 14:50:17.421628 16944 tablet_replica.cc:442] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:17.422894 16979 tablet_replica.cc:439] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a07f10eedf79424d98d3c8544a88b00b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } } }
I20260928 14:50:17.423195 16979 tablet_replica.cc:442] T c45fbbaacc9e464d994fc3273762c3b2 P a07f10eedf79424d98d3c8544a88b00b: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:50:17.426074 16944 tablet_replica.cc:439] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a07f10eedf79424d98d3c8544a88b00b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 } } peers { permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 } } peers { permanent_uuid: "d9a0445446d348efa3edea69e8eb2166" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35515 } } }
I20260928 14:50:17.426355 16944 tablet_replica.cc:442] T c45fbbaacc9e464d994fc3273762c3b2 P 06053ead850b45a2be9da92be62da7a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:17.436682 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.426788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.14.241.195:35515)
0928 14:50:17.426989 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.426996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.428393 (+ 1397us) server_negotiation.cc:407] Connection header received
0928 14:50:17.428683 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.428689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.428826 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.428988 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:17.429883 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.430661 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.434160 (+ 3499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.434678 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.434797 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.435458 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:17.436036 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:17.436154 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.436319 (+ 165us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.436415 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:50:17.453955 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.447140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:36835 (local address 127.14.241.194:34981)
0928 14:50:17.447368 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.447376 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.447971 (+ 595us) server_negotiation.cc:407] Connection header received
0928 14:50:17.448233 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.448237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.448352 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.448431 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:17.449037 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.449848 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.452099 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.452680 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.452780 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.453297 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:17.453415 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:17.453500 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.453633 (+ 133us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.453712 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 14:50:17.454145 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.447026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:34981 (local address 127.14.241.195:36835)
0928 14:50:17.447830 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:17.447860 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:17.448033 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:17.448467 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:17.448473 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:17.448486 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:50:17.448869 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.448883 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.449968 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.449974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.451930 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.451948 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.452852 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.452859 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.453046 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.453125 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:50:17.453524 (+ 399us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:50:17.453535 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 14:50:17.453616 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 14:50:17.453766 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":398,"threads_started":1}
I20260928 14:50:17.463893 16725 raft_consensus.cc:1261] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b [term 1 FOLLOWER]: Refusing update from remote peer d9a0445446d348efa3edea69e8eb2166: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:17.463904 16591 raft_consensus.cc:1261] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5 [term 1 FOLLOWER]: Refusing update from remote peer d9a0445446d348efa3edea69e8eb2166: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:17.464730 16964 consensus_queue.cc:1035] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07f10eedf79424d98d3c8544a88b00b" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:17.465610 16941 consensus_queue.cc:1035] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06053ead850b45a2be9da92be62da7a5" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 42869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:17.504914 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.497747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:55231 (local address 127.14.241.195:35515)
0928 14:50:17.497930 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.497936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.498207 (+ 271us) server_negotiation.cc:407] Connection header received
0928 14:50:17.498518 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.498525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.498627 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.498687 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:17.499633 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.500477 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.502796 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.503461 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.503558 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.504078 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:17.504182 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:17.504242 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.504467 (+ 225us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.504558 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 14:50:17.505795 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.497264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:35515 (local address 127.14.241.194:55231)
0928 14:50:17.498103 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:17.498122 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:17.498323 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:17.498876 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:17.498881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:17.498894 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:50:17.499429 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.499444 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.500601 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.500607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.502610 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:17.502627 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.503639 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.503646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:17.503845 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.503923 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:50:17.504268 (+ 345us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:50:17.504312 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 14:50:17.505106 (+ 794us) client_negotiation.cc:240] Negotiation successful
0928 14:50:17.505263 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":1051,"threads_started":1}
I20260928 14:50:17.543478 16429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:17.543787 16429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:17.547483 16429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41a53cf2606d451f8414c922f4067e1a: Sending DeleteTablet for 3 replicas of tablet 67f43ef5216846bd80e40e461a8bbead
I20260928 14:50:17.548604 16705 tablet_service.cc:1514] Processing DeleteTablet for tablet 67f43ef5216846bd80e40e461a8bbead with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:17 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59760
I20260928 14:50:17.548650 16565 tablet_service.cc:1514] Processing DeleteTablet for tablet 67f43ef5216846bd80e40e461a8bbead with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:17 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:55046
I20260928 14:50:17.556896 16844 tablet_service.cc:1514] Processing DeleteTablet for tablet 67f43ef5216846bd80e40e461a8bbead with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:17 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:44764
I20260928 14:50:17.561915 17005 tablet_replica.cc:330] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b: stopping tablet replica
I20260928 14:50:17.562317 17005 raft_consensus.cc:2229] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:17.562767 17005 raft_consensus.cc:2258] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:17.564543 17005 ts_tablet_manager.cc:1905] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:17.564908 17008 tablet_replica.cc:330] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166: stopping tablet replica
I20260928 14:50:17.565143 17008 raft_consensus.cc:2229] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:50:17.565826 17008 raft_consensus.cc:2258] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:17.567086 17008 ts_tablet_manager.cc:1905] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:17.575363 17008 ts_tablet_manager.cc:1918] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:50:17.575556 17008 log.cc:1196] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/67f43ef5216846bd80e40e461a8bbead
I20260928 14:50:17.576960 17008 ts_tablet_manager.cc:1939] T 67f43ef5216846bd80e40e461a8bbead P d9a0445446d348efa3edea69e8eb2166: Deleting consensus metadata
I20260928 14:50:17.578320 16414 catalog_manager.cc:4841] TS d9a0445446d348efa3edea69e8eb2166 (127.14.241.195:35515): tablet 67f43ef5216846bd80e40e461a8bbead (table test-table [id=6b53194c19e541909d688b797e3da621]) successfully deleted
I20260928 14:50:17.588613 17005 ts_tablet_manager.cc:1918] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:50:17.588824 17005 log.cc:1196] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/67f43ef5216846bd80e40e461a8bbead
I20260928 14:50:17.589313 17005 ts_tablet_manager.cc:1939] T 67f43ef5216846bd80e40e461a8bbead P a07f10eedf79424d98d3c8544a88b00b: Deleting consensus metadata
I20260928 14:50:17.590682 16415 catalog_manager.cc:4841] TS a07f10eedf79424d98d3c8544a88b00b (127.14.241.194:34981): tablet 67f43ef5216846bd80e40e461a8bbead (table test-table [id=6b53194c19e541909d688b797e3da621]) successfully deleted
I20260928 14:50:17.594178 17006 tablet_replica.cc:330] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5: stopping tablet replica
I20260928 14:50:17.594419 17006 raft_consensus.cc:2229] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:17.600361 17006 raft_consensus.cc:2258] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:17.601711 17006 ts_tablet_manager.cc:1905] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 14:50:17 dist-test-slave-vk40 krb5kdc[16384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607016, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:17.609854 17006 ts_tablet_manager.cc:1918] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:50:17.610014 17006 log.cc:1196] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/67f43ef5216846bd80e40e461a8bbead
I20260928 14:50:17.610499 17006 ts_tablet_manager.cc:1939] T 67f43ef5216846bd80e40e461a8bbead P 06053ead850b45a2be9da92be62da7a5: Deleting consensus metadata
I20260928 14:50:17.611574 16416 catalog_manager.cc:4841] TS 06053ead850b45a2be9da92be62da7a5 (127.14.241.193:42869): tablet 67f43ef5216846bd80e40e461a8bbead (table test-table [id=6b53194c19e541909d688b797e3da621]) successfully deleted
I20260928 14:50:17.624508 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.588088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55052 (local address 127.14.241.193:42869)
0928 14:50:17.588383 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.588390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.591948 (+ 3558us) server_negotiation.cc:407] Connection header received
0928 14:50:17.593285 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.593291 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.593384 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.593459 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:17.596393 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.597681 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.600371 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.601066 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.601170 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.616368 (+ 15198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:17.616389 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:17.616408 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:17.616470 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:17.620688 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.621776 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.621787 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.621791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.621963 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.622450 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.622456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.622459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.622896 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:17.623214 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.623818 (+ 604us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.624089 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260928 14:50:17.626732 16631 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55052
I20260928 14:50:17.661954 16971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.637619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56496 (local address 127.14.241.254:41735)
0928 14:50:17.637865 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.637872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.644768 (+ 6896us) server_negotiation.cc:407] Connection header received
0928 14:50:17.645064 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.645071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.645160 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.645233 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:17.647251 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.648448 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.650406 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.651116 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.651214 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.654556 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:17.654564 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:17.654568 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:17.654632 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:17.658561 (+ 3929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.659521 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.659527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.659530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.659697 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.660107 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.660112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.660116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.660543 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:17.660824 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.661137 (+ 313us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.661406 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260928 14:50:17.662635 16429 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56496
I20260928 14:50:17.665264 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 16500
I20260928 14:50:17.684530 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.667007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55056 (local address 127.14.241.193:42869)
0928 14:50:17.667198 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.667204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.667225 (+ 21us) server_negotiation.cc:407] Connection header received
0928 14:50:17.667432 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.667439 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.667527 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.667598 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:17.669400 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.670563 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.672451 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.673159 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.673308 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.677304 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:17.677311 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:17.677317 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:17.677361 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:17.680999 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.681892 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.681899 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.681903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.682070 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.682539 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.682544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.682548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.682939 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:17.683247 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.683832 (+ 585us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.684088 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260928 14:50:17.686008 16631 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55056
W20260928 14:50:17.686698 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 16500: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:50:17.686903 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 16500
I20260928 14:50:17.715444 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 16640
Sep 28 14:50:17 dist-test-slave-vk40 krb5kdc[16384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607016, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:50:17.736203 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.716265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59780 (local address 127.14.241.194:34981)
0928 14:50:17.716502 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.716509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.716806 (+ 297us) server_negotiation.cc:407] Connection header received
0928 14:50:17.716991 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.716997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.717098 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.717218 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:17.718932 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.720170 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.722083 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.722730 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.722885 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.728488 (+ 5603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:17.728505 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:17.728527 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:17.728579 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:17.732780 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.733724 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.733735 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.733739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.733927 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.734380 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.734386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.734389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.734893 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:17.735219 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.735588 (+ 369us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.735825 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260928 14:50:17.737385 16770 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59780
W20260928 14:50:17.738126 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 16640: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:50:17.738307 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 16640
I20260928 14:50:17.763461 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 16779
Sep 28 14:50:17 dist-test-slave-vk40 krb5kdc[16384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607016, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:50:17.779676 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:17.764170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.14.241.195:35515)
0928 14:50:17.764403 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:17.764409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:17.764429 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:50:17.764547 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:17.764554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:17.764636 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:17.764753 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:17.765953 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.767078 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.768235 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:17.769033 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:17.769163 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:17.773085 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:17.773104 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:17.773125 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:17.773174 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:17.776885 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.777566 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.777575 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.777579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.777757 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:17.778129 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:17.778134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:17.778138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:17.778615 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:17.778902 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:17.779073 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 14:50:17.779301 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260928 14:50:17.780424 16909 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44790
W20260928 14:50:17.781083 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 16779: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:50:17.781219 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 16779
I20260928 14:50:17.796051 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 16397
I20260928 14:50:17.797214 16469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:17.910883 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 16397
2026-09-28T14:50:17Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4569 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:50:18 dist-test-slave-vk40 krb5kdc[17016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:50:18 dist-test-slave-vk40 krb5kdc[17016](info): set up 2 sockets
Sep 28 14:50:18 dist-test-slave-vk40 krb5kdc[17016](info): commencing operation
krb5kdc: starting...
W20260928 14:50:18.911439 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:50:19 dist-test-slave-vk40 krb5kdc[17016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:50:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:50:19.092144 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:43897
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:45681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:43897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:19.420749 17029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:19.421062 17029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:19.421178 17029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:19.428534 17029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:19.428648 17029 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 14:50:19.428695 17029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:19.428781 17029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:19.428841 17029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:19.428934 17029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:19.439069 17029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:43897
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:43897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:19.440493 17029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:19.441864 17029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:19.453456 17034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:19.453373 17037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:19.453428 17035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:19.454603 17029 server_base.cc:1034] running on GCE node
I20260928 14:50:19.455354 17029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:19.456801 17029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:19.458014 17029 hybrid_clock.cc:648] HybridClock initialized: now 1790607019457942 us; error 46 us; skew 500 ppm
I20260928 14:50:19.458313 17029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:19 dist-test-slave-vk40 krb5kdc[17016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607019, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:19.461467 17029 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:50:19.462975 17029 webserver.cc:460] Webserver started at http://127.14.241.254:44595/ using document root <none> and password file <none>
I20260928 14:50:19.463658 17029 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:19.463757 17029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:19.464066 17029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:19.467198 17029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "7d56291ab4d54a3182810ea9f337785f"
format_stamp: "Formatted at 2026-09-28 14:50:19 on dist-test-slave-vk40"
server_key: "551b59a34333abfc14ef2697ec8159a9"
server_key_iv: "22930ddef9a656ff9f96e0413d395f66"
server_key_version: "encryptionkey@0"
I20260928 14:50:19.467988 17029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "7d56291ab4d54a3182810ea9f337785f"
format_stamp: "Formatted at 2026-09-28 14:50:19 on dist-test-slave-vk40"
server_key: "551b59a34333abfc14ef2697ec8159a9"
server_key_iv: "22930ddef9a656ff9f96e0413d395f66"
server_key_version: "encryptionkey@0"
I20260928 14:50:19.473415 17029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260928 14:50:19.477007 17044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:19.478436 17029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:50:19.478607 17029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "7d56291ab4d54a3182810ea9f337785f"
format_stamp: "Formatted at 2026-09-28 14:50:19 on dist-test-slave-vk40"
server_key: "551b59a34333abfc14ef2697ec8159a9"
server_key_iv: "22930ddef9a656ff9f96e0413d395f66"
server_key_version: "encryptionkey@0"
I20260928 14:50:19.478837 17029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:19.501168 17029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:19.505694 17029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:19.505996 17029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:19.529100 17029 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:43897
I20260928 14:50:19.529138 17105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:43897 every 8 connection(s)
I20260928 14:50:19.530504 17029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:19.534332 17106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:19.539646 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 17029
I20260928 14:50:19.539848 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:19.540637 15303 external_mini_cluster.cc:1428] Setting key 7f317389691981d63ec50cbdc6ab7383
I20260928 14:50:19.546015 17106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f: Bootstrap starting.
I20260928 14:50:19.549782 17106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:19.551083 17106 log.cc:824] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:19.554611 17106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f: No bootstrap required, opened a new log
Sep 28 14:50:19 dist-test-slave-vk40 krb5kdc[17016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:19.562572 17106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d56291ab4d54a3182810ea9f337785f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43897 } }
I20260928 14:50:19.563035 17106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:19.563141 17106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d56291ab4d54a3182810ea9f337785f, State: Initialized, Role: FOLLOWER
I20260928 14:50:19.563768 17106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d56291ab4d54a3182810ea9f337785f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43897 } }
I20260928 14:50:19.563912 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:19.542758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50284 (local address 127.14.241.254:43897)
0928 14:50:19.543623 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:19.543639 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:19.543685 (+ 46us) server_negotiation.cc:407] Connection header received
0928 14:50:19.545425 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:19.545450 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:19.545875 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:19.546249 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:19.547947 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:19.549315 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:19.551072 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:19.551993 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:19.552184 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:19.556564 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:19.556573 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:19.556589 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:19.556634 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:19.559677 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:19.560406 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:19.560415 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:19.560422 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:19.560599 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:19.560955 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:19.560958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:19.560960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:19.561555 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:19.561865 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:19.562281 (+ 416us) server_negotiation.cc:299] Negotiation successful
0928 14:50:19.562724 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":390,"threads_started":1}
I20260928 14:50:19.564006 17106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:19.564136 17106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:19.564375 17106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:19.566748 17106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d56291ab4d54a3182810ea9f337785f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43897 } }
I20260928 14:50:19.567293 17106 leader_election.cc:302] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d56291ab4d54a3182810ea9f337785f; no voters:
I20260928 14:50:19.567799 17106 leader_election.cc:288] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:19.567925 17111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:19.569000 17111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [term 1 LEADER]: Becoming Leader. State: Replica: 7d56291ab4d54a3182810ea9f337785f, State: Running, Role: LEADER
I20260928 14:50:19.569636 17111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d56291ab4d54a3182810ea9f337785f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43897 } }
I20260928 14:50:19.569818 17106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:19.572602 17113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d56291ab4d54a3182810ea9f337785f. Latest consensus state: current_term: 1 leader_uuid: "7d56291ab4d54a3182810ea9f337785f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d56291ab4d54a3182810ea9f337785f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43897 } } }
I20260928 14:50:19.572999 17113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:19.572572 17112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d56291ab4d54a3182810ea9f337785f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d56291ab4d54a3182810ea9f337785f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43897 } } }
I20260928 14:50:19.573503 17112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:19.573884 17120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:19.579650 17120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:19.589610 17120 catalog_manager.cc:1347] Generated new cluster ID: 11778437f0f143adaca3e1f88e306e64
I20260928 14:50:19.589738 17120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:19.621035 17120 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:19.622946 17120 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:19.640756 17120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f: Generated new TSK 0
I20260928 14:50:19.642061 17120 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:50:19.702059 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:43897
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:45681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:20.030311 17132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:20.030629 17132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:20.030781 17132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:20.038194 17132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:20.038336 17132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:20.038471 17132 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 14:50:20.038539 17132 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:50:20.038645 17132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:20.048600 17132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.241.254:43897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:20.049969 17132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:20.051340 17132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:20.064459 17137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:20.064484 17138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:20.064700 17140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:20.065539 17132 server_base.cc:1034] running on GCE node
I20260928 14:50:20.066169 17132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:20.067163 17132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:20.068413 17132 hybrid_clock.cc:648] HybridClock initialized: now 1790607020068340 us; error 42 us; skew 500 ppm
I20260928 14:50:20.068710 17132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:20 dist-test-slave-vk40 krb5kdc[17016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607020, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:20.072009 17132 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:50:20.073536 17132 webserver.cc:460] Webserver started at http://127.14.241.193:39709/ using document root <none> and password file <none>
I20260928 14:50:20.074304 17132 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:20.074424 17132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:20.074697 17132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:20.077682 17132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "882926d3969841bf8e0cb9fcb4172e30"
format_stamp: "Formatted at 2026-09-28 14:50:20 on dist-test-slave-vk40"
server_key: "77d4e487e1c95525929c684d14548daf"
server_key_iv: "22f7ffa219d310eeb7dafffc3c3f96e4"
server_key_version: "encryptionkey@0"
I20260928 14:50:20.078419 17132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "882926d3969841bf8e0cb9fcb4172e30"
format_stamp: "Formatted at 2026-09-28 14:50:20 on dist-test-slave-vk40"
server_key: "77d4e487e1c95525929c684d14548daf"
server_key_iv: "22f7ffa219d310eeb7dafffc3c3f96e4"
server_key_version: "encryptionkey@0"
I20260928 14:50:20.083969 17132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260928 14:50:20.087669 17147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:20.089164 17132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:20.089314 17132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "882926d3969841bf8e0cb9fcb4172e30"
format_stamp: "Formatted at 2026-09-28 14:50:20 on dist-test-slave-vk40"
server_key: "77d4e487e1c95525929c684d14548daf"
server_key_iv: "22f7ffa219d310eeb7dafffc3c3f96e4"
server_key_version: "encryptionkey@0"
I20260928 14:50:20.089504 17132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:20.107071 17132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:20.111438 17132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:20.111696 17132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:20.114352 17132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:20.114460 17132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:20.114575 17132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:20.114642 17132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:50:20 dist-test-slave-vk40 krb5kdc[17016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607020, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:20.140618 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:20.115061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:40561 (local address 127.14.241.254:43897)
0928 14:50:20.117847 (+ 2786us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:20.117852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:20.117870 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:50:20.120171 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:20.120178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:20.120313 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:20.120447 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:20.123279 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.124126 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.127158 (+ 3032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.127686 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.127831 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:20.133175 (+ 5344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:20.133181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:20.133184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:20.133224 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:20.136504 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:20.137679 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:20.137686 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:20.137689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:20.137865 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:20.138470 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:20.138475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:20.138479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:20.138860 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:20.139005 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:20.139961 (+ 956us) server_negotiation.cc:299] Negotiation successful
0928 14:50:20.140225 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2681,"thread_start_us":2569,"threads_started":1}
I20260928 14:50:20.142045 17157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:20.115909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43897 (local address 127.14.241.193:40561)
0928 14:50:20.116959 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:20.117014 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:20.119752 (+ 2738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:20.120924 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:20.120939 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:20.121923 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:20.123048 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:20.123078 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.124271 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.124298 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:20.126981 (+ 2683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:20.127002 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.128019 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.128027 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:20.128436 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:20.129361 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:20.129399 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:20.132821 (+ 3422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:20.136680 (+ 3859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:20.136691 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:20.136720 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:20.137513 (+ 793us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:20.137991 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:20.137997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:20.138001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:20.138315 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:20.138994 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:20.139002 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:20.139570 (+ 568us) client_negotiation.cc:769] Sending connection context
0928 14:50:20.139841 (+ 271us) client_negotiation.cc:240] Negotiation successful
0928 14:50:20.140513 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":428,"threads_started":1}
I20260928 14:50:20.154815 17132 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:40707
I20260928 14:50:20.154883 17267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:40707 every 8 connection(s)
I20260928 14:50:20.156351 17132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:20.158622 17268 heartbeater.cc:347] Connected to a master server at 127.14.241.254:43897
I20260928 14:50:20.158851 17268 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:20.159662 17268 heartbeater.cc:511] Master 127.14.241.254:43897 requested a full tablet report, sending...
I20260928 14:50:20.160962 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:20.148091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:33577 (local address 127.14.241.254:43897)
0928 14:50:20.148257 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:20.148264 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:20.148936 (+ 672us) server_negotiation.cc:407] Connection header received
0928 14:50:20.149097 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:20.149103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:20.149173 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:20.149251 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:20.150524 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.151275 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.152561 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.153088 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.153196 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:20.155506 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:20.155511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:20.155514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:20.155547 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:20.158291 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:20.159030 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:20.159035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:20.159039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:20.159202 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:20.159536 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:20.159541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:20.159544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:20.159898 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:20.160049 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:20.160454 (+ 405us) server_negotiation.cc:299] Negotiation successful
0928 14:50:20.160653 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:50:20.160984 17257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:20.148159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43897 (local address 127.14.241.193:33577)
0928 14:50:20.148819 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:20.148834 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:20.148984 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:20.149289 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:20.149293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:20.149893 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:20.150377 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:20.150392 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.151396 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.151402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:20.152436 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:20.152448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.153265 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.153271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:20.153435 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:20.154033 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:20.154057 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:20.155333 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:20.158431 (+ 3098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:20.158437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:20.158441 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:20.158928 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:20.159302 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:20.159306 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:20.159308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:20.159434 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:20.160003 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:20.160006 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:20.160162 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 14:50:20.160269 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 14:50:20.160595 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":417,"threads_started":1}
I20260928 14:50:20.161759 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 17132
I20260928 14:50:20.161932 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:50:20.161939 17061 ts_manager.cc:194] Registered new tserver with Master: 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707)
I20260928 14:50:20.162537 15303 external_mini_cluster.cc:1428] Setting key 5dfeceadcbe37f0fb8b642673e7ea785
I20260928 14:50:20.163617 17061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:40561
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:50:20.204412 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:43897
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:45681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:20.538247 17272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:20.538544 17272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:20.538691 17272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:20.546269 17272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:20.546406 17272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:20.546526 17272 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 14:50:20.546591 17272 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:50:20.546712 17272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:20.556667 17272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.241.254:43897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:20.557977 17272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:20.559319 17272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:20.572446 17278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:20.572500 17277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:20.572640 17280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:20.573261 17272 server_base.cc:1034] running on GCE node
I20260928 14:50:20.573943 17272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:20.574900 17272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:20.576098 17272 hybrid_clock.cc:648] HybridClock initialized: now 1790607020576037 us; error 52 us; skew 500 ppm
I20260928 14:50:20.576457 17272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:20 dist-test-slave-vk40 krb5kdc[17016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607020, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:20.579561 17272 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:50:20.581143 17272 webserver.cc:460] Webserver started at http://127.14.241.194:39695/ using document root <none> and password file <none>
I20260928 14:50:20.581828 17272 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:20.581939 17272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:20.582237 17272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:20.585290 17272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "376cfe628b4343829c7fe5d4f29e756c"
format_stamp: "Formatted at 2026-09-28 14:50:20 on dist-test-slave-vk40"
server_key: "6238fd24321f3b45831f584a82479a42"
server_key_iv: "dba825084ea634288c63dc46777055f2"
server_key_version: "encryptionkey@0"
I20260928 14:50:20.586030 17272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "376cfe628b4343829c7fe5d4f29e756c"
format_stamp: "Formatted at 2026-09-28 14:50:20 on dist-test-slave-vk40"
server_key: "6238fd24321f3b45831f584a82479a42"
server_key_iv: "dba825084ea634288c63dc46777055f2"
server_key_version: "encryptionkey@0"
I20260928 14:50:20.591403 17272 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:50:20.595254 17287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:20.596828 17272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:20.597002 17272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "376cfe628b4343829c7fe5d4f29e756c"
format_stamp: "Formatted at 2026-09-28 14:50:20 on dist-test-slave-vk40"
server_key: "6238fd24321f3b45831f584a82479a42"
server_key_iv: "dba825084ea634288c63dc46777055f2"
server_key_version: "encryptionkey@0"
I20260928 14:50:20.597199 17272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:20.615939 17272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:20.619982 17272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:20.620260 17272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:20.622953 17272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:20.623070 17272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:20.623184 17272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:20.623270 17272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:50:20 dist-test-slave-vk40 krb5kdc[17016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607020, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:20.646520 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:20.623481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:59449 (local address 127.14.241.254:43897)
0928 14:50:20.623685 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:20.623692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:20.625204 (+ 1512us) server_negotiation.cc:407] Connection header received
0928 14:50:20.627345 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:20.627352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:20.627442 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:20.627528 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:20.629556 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.630739 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.633360 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.634048 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.634189 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:20.639007 (+ 4818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:20.639016 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:20.639021 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:20.639073 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:20.642652 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:20.643546 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:20.643553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:20.643557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:20.643717 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:20.644308 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:20.644315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:20.644318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:20.644629 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:20.644772 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:20.645881 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0928 14:50:20.646154 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 14:50:20.648801 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:20.624211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43897 (local address 127.14.241.194:59449)
0928 14:50:20.625043 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:20.625083 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:20.627032 (+ 1949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:20.627782 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:20.627790 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:20.628360 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:20.629297 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:20.629322 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.630934 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.630940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:20.633170 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:20.633187 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.634318 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.634329 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:20.634663 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:20.635555 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:20.635594 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:20.638702 (+ 3108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:20.642824 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:20.642832 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:20.642847 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:20.643417 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:20.643833 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:20.643839 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:20.643843 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:20.644151 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:20.644788 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:20.644797 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:20.645336 (+ 539us) client_negotiation.cc:769] Sending connection context
0928 14:50:20.646620 (+ 1284us) client_negotiation.cc:240] Negotiation successful
0928 14:50:20.647159 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":328,"threads_started":1}
I20260928 14:50:20.673341 17272 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:33021
I20260928 14:50:20.673370 17406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:33021 every 8 connection(s)
I20260928 14:50:20.674907 17272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:20.677376 17407 heartbeater.cc:347] Connected to a master server at 127.14.241.254:43897
I20260928 14:50:20.677587 17407 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:20.678313 17407 heartbeater.cc:511] Master 127.14.241.254:43897 requested a full tablet report, sending...
I20260928 14:50:20.679708 17061 ts_manager.cc:194] Registered new tserver with Master: 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194:33021)
I20260928 14:50:20.680713 17061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:59449
I20260928 14:50:20.681596 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 17272
I20260928 14:50:20.681742 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:20.667233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:50621 (local address 127.14.241.254:43897)
0928 14:50:20.667428 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:20.667434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:20.668776 (+ 1342us) server_negotiation.cc:407] Connection header received
0928 14:50:20.668916 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:20.668924 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:20.669017 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:20.669146 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:20.670079 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.671188 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.673316 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.673914 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.674016 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:20.676267 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:20.676274 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:20.676305 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:20.676357 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:20.679002 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:20.679693 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:20.679697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:20.679700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:20.679855 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:20.680250 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:20.680254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:20.680256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:20.680627 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:20.680743 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:20.681184 (+ 441us) server_negotiation.cc:299] Negotiation successful
0928 14:50:20.681441 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 14:50:20.681761 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:50:20.681798 17400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:20.667375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43897 (local address 127.14.241.194:50621)
0928 14:50:20.668644 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:20.668660 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:20.668795 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:20.669179 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:20.669183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:20.669638 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:20.669943 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:20.669953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.671401 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.671407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:20.673156 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:20.673175 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:20.674105 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:20.674112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:20.674285 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:20.674897 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:20.674920 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:20.676039 (+ 1119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:20.679125 (+ 3086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:20.679131 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:20.679135 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:20.679583 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:20.679984 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:20.679989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:20.679992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:20.680142 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:20.680739 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:20.680744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:20.680932 (+ 188us) client_negotiation.cc:769] Sending connection context
0928 14:50:20.681081 (+ 149us) client_negotiation.cc:240] Negotiation successful
0928 14:50:20.681409 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1132,"thread_start_us":833,"threads_started":1}
I20260928 14:50:20.682304 15303 external_mini_cluster.cc:1428] Setting key 4812d70e1835116fa9357260a86db068
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:50:20.723171 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:43897
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:45681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:21.049033 17411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:21.049333 17411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:21.049480 17411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:21.056854 17411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:21.056975 17411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:21.057101 17411 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 14:50:21.057163 17411 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:50:21.057268 17411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:21.067191 17411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.241.254:43897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:21.068526 17411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:21.069851 17411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:21.082506 17419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:21.082558 17417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:21.082715 17416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:21.083442 17411 server_base.cc:1034] running on GCE node
I20260928 14:50:21.084067 17411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:21.085083 17411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:21.086275 17411 hybrid_clock.cc:648] HybridClock initialized: now 1790607021086214 us; error 59 us; skew 500 ppm
I20260928 14:50:21.086591 17411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:21 dist-test-slave-vk40 krb5kdc[17016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607021, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:21.089797 17411 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:50:21.091302 17411 webserver.cc:460] Webserver started at http://127.14.241.195:39465/ using document root <none> and password file <none>
I20260928 14:50:21.091985 17411 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:21.092088 17411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:21.092437 17411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:21.095651 17411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5"
format_stamp: "Formatted at 2026-09-28 14:50:21 on dist-test-slave-vk40"
server_key: "a9e515dd79573caf004ea511c56381b8"
server_key_iv: "97214c9ffc8e6a88ea9c5e7c3fb3ef85"
server_key_version: "encryptionkey@0"
I20260928 14:50:21.096491 17411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5"
format_stamp: "Formatted at 2026-09-28 14:50:21 on dist-test-slave-vk40"
server_key: "a9e515dd79573caf004ea511c56381b8"
server_key_iv: "97214c9ffc8e6a88ea9c5e7c3fb3ef85"
server_key_version: "encryptionkey@0"
I20260928 14:50:21.102005 17411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:50:21.105930 17426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:21.107446 17411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:50:21.107620 17411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5"
format_stamp: "Formatted at 2026-09-28 14:50:21 on dist-test-slave-vk40"
server_key: "a9e515dd79573caf004ea511c56381b8"
server_key_iv: "97214c9ffc8e6a88ea9c5e7c3fb3ef85"
server_key_version: "encryptionkey@0"
I20260928 14:50:21.107813 17411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:21.121358 17411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:21.125623 17411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:21.125897 17411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:21.128636 17411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:21.128741 17411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:21.128847 17411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:21.128928 17411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:50:21 dist-test-slave-vk40 krb5kdc[17016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607021, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:21.154234 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.129256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:41861 (local address 127.14.241.254:43897)
0928 14:50:21.129449 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.129455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.130898 (+ 1443us) server_negotiation.cc:407] Connection header received
0928 14:50:21.133735 (+ 2837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.133742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.133842 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.133962 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:21.136704 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.137681 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.140639 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.141166 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.141306 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.146105 (+ 4799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:21.146114 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:21.146120 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:21.146216 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:21.149926 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:21.151265 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:21.151271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:21.151275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:21.151461 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:21.152085 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:21.152109 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:21.152112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:21.152561 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:21.152725 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.153619 (+ 894us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.153873 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 14:50:21.155344 17435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.129868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43897 (local address 127.14.241.195:41861)
0928 14:50:21.130694 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.130761 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.133330 (+ 2569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.134278 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.134292 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.135248 (+ 956us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:21.136400 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.136432 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.137891 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.137897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.140450 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.140469 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.141455 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.141462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.141822 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.142733 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:21.142776 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:21.145778 (+ 3002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:21.150098 (+ 4320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:21.150114 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:21.150139 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:21.151113 (+ 974us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:21.151615 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:21.151620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:21.151624 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:21.151949 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:21.152801 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:21.152809 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:21.153202 (+ 393us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.153456 (+ 254us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.154028 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":336,"threads_started":1}
I20260928 14:50:21.166931 17268 heartbeater.cc:503] Master 127.14.241.254:43897 was elected leader, sending a full tablet report...
I20260928 14:50:21.179226 17411 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:35457
I20260928 14:50:21.179246 17545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:35457 every 8 connection(s)
I20260928 14:50:21.180847 17411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:21.181249 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 17411
I20260928 14:50:21.181418 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:50:21.182013 15303 external_mini_cluster.cc:1428] Setting key 83cf3ff7537d16852a648f3bef49ab92
I20260928 14:50:21.183326 17546 heartbeater.cc:347] Connected to a master server at 127.14.241.254:43897
I20260928 14:50:21.183660 17546 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:21.184681 17546 heartbeater.cc:511] Master 127.14.241.254:43897 requested a full tablet report, sending...
I20260928 14:50:21.185614 17521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.169338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43897 (local address 127.14.241.195:53367)
0928 14:50:21.169979 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.169999 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.170131 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.170538 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.170546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.171058 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:21.171434 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.171449 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.173047 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.173052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.175001 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.175018 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.176259 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.176267 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.176475 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.177259 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:21.177286 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:21.178346 (+ 1060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:21.182196 (+ 3850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:21.182203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:21.182207 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:21.182798 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:21.183279 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:21.183285 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:21.183289 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:21.183459 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:21.184229 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:21.184234 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:21.184531 (+ 297us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.184700 (+ 169us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.185088 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":25,"thread_start_us":362,"threads_started":1}
I20260928 14:50:21.186486 17060 ts_manager.cc:194] Registered new tserver with Master: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457)
I20260928 14:50:21.187112 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.169249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:53367 (local address 127.14.241.254:43897)
0928 14:50:21.169450 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.169457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.170076 (+ 619us) server_negotiation.cc:407] Connection header received
0928 14:50:21.170279 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.170284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.170366 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.170491 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:21.171613 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.172841 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.175216 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.175845 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.175980 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.178597 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:21.178604 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:21.178609 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:21.178650 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:21.182016 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:21.182966 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:21.182972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:21.182976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:21.183150 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:21.183582 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:21.183588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:21.183591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:21.183970 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:21.184162 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.185940 (+ 1778us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.186681 (+ 741us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:50:21.187577 17060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:41861
I20260928 14:50:21.197837 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:50:21 dist-test-slave-vk40 krb5kdc[17016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:50:21 dist-test-slave-vk40 krb5kdc[17016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:21.241705 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.228785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50286 (local address 127.14.241.254:43897)
0928 14:50:21.229016 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.229023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.229135 (+ 112us) server_negotiation.cc:407] Connection header received
0928 14:50:21.229319 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.229325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.229396 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.229481 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:21.230765 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.231514 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.232776 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.233202 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.233311 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.236934 (+ 3623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:21.236940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:21.236943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:21.236980 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:21.239229 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:21.239836 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:21.239848 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:21.239851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:21.239980 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:21.240326 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:21.240330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:21.240334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:21.240608 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:21.240858 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.241133 (+ 275us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.241348 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 14:50:21.245805 17060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:50:21.252699 17060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:21.275280 17559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.264773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:35457 (local address 127.0.0.1:59142)
0928 14:50:21.265556 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.265600 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.265783 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.267306 (+ 1523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.267312 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.267353 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:21.267743 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.267757 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.269847 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.269852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.272134 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.272152 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.274342 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.274347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.274506 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.274541 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.274662 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.274873 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":351,"threads_started":1}
I20260928 14:50:21.275274 17560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.264907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59142 (local address 127.14.241.195:35457)
0928 14:50:21.266668 (+ 1761us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.266676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.266724 (+ 48us) server_negotiation.cc:407] Connection header received
0928 14:50:21.266837 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.266846 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.267100 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.267284 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:21.267988 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.269660 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.272324 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.273914 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.274084 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.274216 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.274743 (+ 527us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.274918 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1606,"thread_start_us":1434,"threads_started":1}
I20260928 14:50:21.276603 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.267220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39244 (local address 127.14.241.193:40707)
0928 14:50:21.268501 (+ 1281us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.268510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.268535 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:50:21.268622 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.268629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.268863 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.269691 (+ 828us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:21.269910 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.271504 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.274002 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.275498 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.275684 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.275853 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.276101 (+ 248us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.276300 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1128,"thread_start_us":967,"threads_started":1}
I20260928 14:50:21.276614 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.266985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:40707 (local address 127.0.0.1:39244)
0928 14:50:21.267923 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.267938 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.268027 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.269059 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.269063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.269087 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:21.269456 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.269467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.271641 (+ 2174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.271648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.273817 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.273833 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.275746 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.275753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.275930 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.275954 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.276032 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.276166 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"thread_start_us":676,"threads_started":1}
I20260928 14:50:21.279563 17476 tablet_service.cc:1467] Processing CreateTablet for tablet 3d75818c49a4472d8b792d2b0edc43ee (DEFAULT_TABLE table=test-table [id=37ab10d2c1bc4dd2a902a6272f4fb885]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:21.280951 17561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.265127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59040 (local address 127.14.241.194:33021)
0928 14:50:21.272091 (+ 6964us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.272100 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.272128 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:50:21.272237 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.272245 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.272474 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.273838 (+ 1364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:21.274029 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.275571 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.279019 (+ 3448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.279923 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.280079 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.280249 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.280510 (+ 261us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.280670 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6807,"thread_start_us":6664,"threads_started":1}
I20260928 14:50:21.280951 17562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.264780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:33021 (local address 127.0.0.1:59040)
0928 14:50:21.267015 (+ 2235us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.267032 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.267137 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.272718 (+ 5581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.272723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.272750 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:21.273375 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.273387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.276033 (+ 2646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.276039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.278820 (+ 2781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.278841 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.280121 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.280128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.280332 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.280358 (+ 26us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.280437 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.280571 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2122,"mutex_wait_us":523,"thread_start_us":1358,"threads_started":1}
I20260928 14:50:21.281157 17476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d75818c49a4472d8b792d2b0edc43ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:21.283986 17198 tablet_service.cc:1467] Processing CreateTablet for tablet 3d75818c49a4472d8b792d2b0edc43ee (DEFAULT_TABLE table=test-table [id=37ab10d2c1bc4dd2a902a6272f4fb885]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:21.285602 17198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d75818c49a4472d8b792d2b0edc43ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:21.285140 17337 tablet_service.cc:1467] Processing CreateTablet for tablet 3d75818c49a4472d8b792d2b0edc43ee (DEFAULT_TABLE table=test-table [id=37ab10d2c1bc4dd2a902a6272f4fb885]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:21.286440 17337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d75818c49a4472d8b792d2b0edc43ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:21.293010 17565 tablet_bootstrap.cc:492] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Bootstrap starting.
I20260928 14:50:21.296203 17565 tablet_bootstrap.cc:654] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:21.297355 17565 log.cc:824] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:21.300150 17567 tablet_bootstrap.cc:492] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: Bootstrap starting.
I20260928 14:50:21.300840 17565 tablet_bootstrap.cc:492] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: No bootstrap required, opened a new log
I20260928 14:50:21.301254 17565 ts_tablet_manager.cc:1397] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 14:50:21.301783 17568 tablet_bootstrap.cc:492] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: Bootstrap starting.
I20260928 14:50:21.303691 17567 tablet_bootstrap.cc:654] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:21.304493 17568 tablet_bootstrap.cc:654] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:21.304963 17567 log.cc:824] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:21.305400 17568 log.cc:824] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:21.308251 17567 tablet_bootstrap.cc:492] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: No bootstrap required, opened a new log
I20260928 14:50:21.308568 17567 ts_tablet_manager.cc:1397] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260928 14:50:21.311342 17565 raft_consensus.cc:348] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } }
I20260928 14:50:21.311838 17565 raft_consensus.cc:374] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:21.311969 17565 raft_consensus.cc:729] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, State: Initialized, Role: FOLLOWER
I20260928 14:50:21.312626 17565 consensus_queue.cc:260] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } }
I20260928 14:50:21.313196 17568 tablet_bootstrap.cc:492] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: No bootstrap required, opened a new log
I20260928 14:50:21.313535 17568 ts_tablet_manager.cc:1397] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.009s
I20260928 14:50:21.313870 17546 heartbeater.cc:503] Master 127.14.241.254:43897 was elected leader, sending a full tablet report...
I20260928 14:50:21.314477 17565 ts_tablet_manager.cc:1428] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260928 14:50:21.318321 17567 raft_consensus.cc:348] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } }
I20260928 14:50:21.318678 17567 raft_consensus.cc:374] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:21.318778 17567 raft_consensus.cc:729] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 882926d3969841bf8e0cb9fcb4172e30, State: Initialized, Role: FOLLOWER
I20260928 14:50:21.319383 17567 consensus_queue.cc:260] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } }
I20260928 14:50:21.321193 17567 ts_tablet_manager.cc:1428] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260928 14:50:21.323458 17568 raft_consensus.cc:348] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } }
I20260928 14:50:21.323836 17568 raft_consensus.cc:374] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:21.323935 17568 raft_consensus.cc:729] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 376cfe628b4343829c7fe5d4f29e756c, State: Initialized, Role: FOLLOWER
I20260928 14:50:21.324573 17568 consensus_queue.cc:260] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } }
I20260928 14:50:21.325583 17407 heartbeater.cc:503] Master 127.14.241.254:43897 was elected leader, sending a full tablet report...
I20260928 14:50:21.326099 17568 ts_tablet_manager.cc:1428] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260928 14:50:21.389137 17571 raft_consensus.cc:482] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:21.389462 17571 raft_consensus.cc:504] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } }
I20260928 14:50:21.390959 17573 raft_consensus.cc:482] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:21.391392 17571 leader_election.cc:288] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194:33021), 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707)
I20260928 14:50:21.391265 17573 raft_consensus.cc:504] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } }
I20260928 14:50:21.399307 17573 leader_election.cc:288] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457), 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707)
I20260928 14:50:21.400141 17561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.391654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:55111 (local address 127.14.241.194:33021)
0928 14:50:21.391853 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.391859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.391879 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:50:21.392084 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.392090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.392167 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.392255 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:21.392922 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.394091 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.397182 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.398585 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.398776 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.398871 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.399663 (+ 792us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.399827 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 14:50:21.400724 17435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.391571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:33021 (local address 127.14.241.195:55111)
0928 14:50:21.391763 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.391784 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.391912 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.392449 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.392454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.392481 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:21.392703 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.392717 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.394764 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.394770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.396922 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.396943 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.398983 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.398990 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.399232 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.399262 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.399354 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.400411 (+ 1057us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260928 14:50:21.402194 17357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d75818c49a4472d8b792d2b0edc43ee" candidate_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "376cfe628b4343829c7fe5d4f29e756c" is_pre_election: true
I20260928 14:50:21.402745 17357 raft_consensus.cc:2454] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 in term 0.
I20260928 14:50:21.403786 17430 leader_election.cc:302] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, 376cfe628b4343829c7fe5d4f29e756c; no voters:
I20260928 14:50:21.404512 17571 raft_consensus.cc:2781] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:50:21.404739 17571 raft_consensus.cc:482] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:50:21.404925 17571 raft_consensus.cc:3037] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:21.408454 17571 raft_consensus.cc:504] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } }
W20260928 14:50:21.408725 17269 tablet.cc:2369] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:50:21.409025 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.392264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:35271 (local address 127.14.241.193:40707)
0928 14:50:21.392517 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.392523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.393215 (+ 692us) server_negotiation.cc:407] Connection header received
0928 14:50:21.393537 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.393542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.393638 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.393707 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:21.401421 (+ 7714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.402619 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.406380 (+ 3761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.407775 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.407915 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.408002 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.408566 (+ 564us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.408718 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":93}
I20260928 14:50:21.409801 17357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d75818c49a4472d8b792d2b0edc43ee" candidate_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "376cfe628b4343829c7fe5d4f29e756c"
I20260928 14:50:21.409916 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.391919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:40707 (local address 127.14.241.195:35271)
0928 14:50:21.393118 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.393135 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.393317 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.400047 (+ 6730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.400052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.400095 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:21.400480 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.400494 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.402874 (+ 2380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.402880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.406097 (+ 3217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.406111 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.408176 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.408182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.408368 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.408402 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.409426 (+ 1024us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.409583 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"thread_start_us":894,"threads_started":1}
I20260928 14:50:21.410060 17357 raft_consensus.cc:3037] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:21.410914 17218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d75818c49a4472d8b792d2b0edc43ee" candidate_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882926d3969841bf8e0cb9fcb4172e30" is_pre_election: true
I20260928 14:50:21.411489 17218 raft_consensus.cc:2454] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 in term 0.
I20260928 14:50:21.413097 17560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.394458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:34817 (local address 127.14.241.195:35457)
0928 14:50:21.394640 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.394645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.405685 (+ 11040us) server_negotiation.cc:407] Connection header received
0928 14:50:21.406149 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.406154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.406227 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.406284 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:21.407066 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.408112 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.410711 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.411821 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.411957 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.412029 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.412732 (+ 703us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.412863 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:50:21.413282 17218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d75818c49a4472d8b792d2b0edc43ee" candidate_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882926d3969841bf8e0cb9fcb4172e30"
I20260928 14:50:21.413481 17218 raft_consensus.cc:3037] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:21.413810 17576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.393515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:35457 (local address 127.14.241.194:34817)
0928 14:50:21.405585 (+ 12070us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.405604 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.405989 (+ 385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.406479 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.406484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.406510 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:21.406861 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.406873 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.408308 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.408313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.410519 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.410537 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.412251 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.412258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.412545 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.412577 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.413255 (+ 678us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.413406 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11931,"thread_start_us":12175,"threads_started":1}
I20260928 14:50:21.414233 17575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.393441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:40707 (local address 127.14.241.194:41313)
0928 14:50:21.404527 (+ 11086us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.404548 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.404679 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.405789 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.405794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.405821 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:21.406241 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.406252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.408182 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.408188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.411478 (+ 3290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.411492 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.413198 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.413205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.413377 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.413409 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.413472 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.413611 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10955,"thread_start_us":11320,"threads_started":1}
I20260928 14:50:21.414601 17577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.394030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:41313 (local address 127.14.241.193:40707)
0928 14:50:21.405459 (+ 11429us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.405466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.405488 (+ 22us) server_negotiation.cc:407] Connection header received
0928 14:50:21.405552 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.405556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.405631 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.406388 (+ 757us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:21.406679 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.408036 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.411707 (+ 3671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.412971 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.413958 (+ 987us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.414035 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.414100 (+ 65us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.414238 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":11277,"thread_start_us":12146,"threads_started":1}
I20260928 14:50:21.414723 17496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d75818c49a4472d8b792d2b0edc43ee" candidate_uuid: "376cfe628b4343829c7fe5d4f29e756c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" is_pre_election: true
I20260928 14:50:21.415197 17496 raft_consensus.cc:2379] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 376cfe628b4343829c7fe5d4f29e756c in current term 1: Already voted for candidate 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 in this term.
I20260928 14:50:21.415112 17217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d75818c49a4472d8b792d2b0edc43ee" candidate_uuid: "376cfe628b4343829c7fe5d4f29e756c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882926d3969841bf8e0cb9fcb4172e30" is_pre_election: true
I20260928 14:50:21.415285 17357 raft_consensus.cc:2454] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 in term 1.
I20260928 14:50:21.415751 17430 leader_election.cc:302] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, 376cfe628b4343829c7fe5d4f29e756c; no voters:
I20260928 14:50:21.416173 17291 leader_election.cc:302] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 376cfe628b4343829c7fe5d4f29e756c; no voters: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, 882926d3969841bf8e0cb9fcb4172e30
I20260928 14:50:21.416388 17571 leader_election.cc:288] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [CANDIDATE]: Term 1 election: Requested vote from peers 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194:33021), 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707)
I20260928 14:50:21.416589 17571 raft_consensus.cc:2781] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:21.416702 17571 raft_consensus.cc:686] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 LEADER]: Becoming Leader. State: Replica: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, State: Running, Role: LEADER
I20260928 14:50:21.416929 17573 raft_consensus.cc:2726] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260928 14:50:21.417198 17571 consensus_queue.cc:237] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } }
I20260928 14:50:21.417392 17218 raft_consensus.cc:2454] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 in term 1.
I20260928 14:50:21.422297 17060 catalog_manager.cc:5495] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195). New cstate: current_term: 1 leader_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } health_report { overall_health: UNKNOWN } } }
W20260928 14:50:21.427279 17408 tablet.cc:2369] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:50:21.438462 17547 tablet.cc:2369] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 14:50:21 dist-test-slave-vk40 krb5kdc[17016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607019, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:21.499176 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.486705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43897 (local address 127.0.0.1:50292)
0928 14:50:21.487319 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.487338 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.487473 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.487774 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.487777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.488505 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:21.488770 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.488780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.489785 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.489789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.490769 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.490779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.491512 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.491516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.491643 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.492276 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:21.492329 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:21.494467 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:21.496977 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:21.496987 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:21.496993 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:21.497386 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:21.497690 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:21.497693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:21.497695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:21.497822 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:21.498265 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:21.498272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:21.498435 (+ 163us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.498562 (+ 127us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.498774 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":344,"threads_started":1}
I20260928 14:50:21.499215 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.486636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.14.241.254:43897)
0928 14:50:21.486861 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.486868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.487428 (+ 560us) server_negotiation.cc:407] Connection header received
0928 14:50:21.487566 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.487570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.487644 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.487729 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:21.488917 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.489665 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.490909 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.491355 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.491452 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.494623 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:21.494628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:21.494631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:21.494660 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:21.496868 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:21.497481 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:21.497485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:21.497487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:21.497605 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:21.497903 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:21.497906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:21.497908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:21.498166 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:21.498272 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.498648 (+ 376us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.498894 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":74}
I20260928 14:50:21.504441 17060 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:50292:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 14:50:21.505460 17060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:21.511833 17337 tablet_service.cc:1467] Processing CreateTablet for tablet 714b63b0e7e14b21aa25ee2ece50ddcc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff8c8aa374f94709a631f09764a5669a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:50:21.511804 17198 tablet_service.cc:1467] Processing CreateTablet for tablet 714b63b0e7e14b21aa25ee2ece50ddcc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff8c8aa374f94709a631f09764a5669a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:50:21.512164 17476 tablet_service.cc:1467] Processing CreateTablet for tablet 714b63b0e7e14b21aa25ee2ece50ddcc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff8c8aa374f94709a631f09764a5669a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:50:21.512847 17198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714b63b0e7e14b21aa25ee2ece50ddcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:21.512862 17337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714b63b0e7e14b21aa25ee2ece50ddcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:21.512931 17476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714b63b0e7e14b21aa25ee2ece50ddcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:21.518718 17567 tablet_bootstrap.cc:492] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: Bootstrap starting.
I20260928 14:50:21.519975 17568 tablet_bootstrap.cc:492] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c: Bootstrap starting.
I20260928 14:50:21.520164 17567 tablet_bootstrap.cc:654] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:21.520464 17565 tablet_bootstrap.cc:492] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Bootstrap starting.
I20260928 14:50:21.521596 17568 tablet_bootstrap.cc:654] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:21.521912 17565 tablet_bootstrap.cc:654] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:21.522689 17567 tablet_bootstrap.cc:492] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: No bootstrap required, opened a new log
I20260928 14:50:21.522840 17567 ts_tablet_manager.cc:1397] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260928 14:50:21.523555 17567 raft_consensus.cc:348] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:21.523759 17567 raft_consensus.cc:374] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:21.523838 17568 tablet_bootstrap.cc:492] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c: No bootstrap required, opened a new log
I20260928 14:50:21.523849 17565 tablet_bootstrap.cc:492] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: No bootstrap required, opened a new log
I20260928 14:50:21.523852 17567 raft_consensus.cc:729] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 882926d3969841bf8e0cb9fcb4172e30, State: Initialized, Role: FOLLOWER
I20260928 14:50:21.523967 17565 ts_tablet_manager.cc:1397] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260928 14:50:21.523974 17568 ts_tablet_manager.cc:1397] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260928 14:50:21.524108 17567 consensus_queue.cc:260] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:21.524682 17567 ts_tablet_manager.cc:1428] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:50:21.524577 17572 tablet_replica.cc:439] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } }
I20260928 14:50:21.524712 17565 raft_consensus.cc:348] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:21.524712 17568 raft_consensus.cc:348] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:21.524945 17568 raft_consensus.cc:374] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:21.524957 17565 raft_consensus.cc:374] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:21.524976 17572 tablet_replica.cc:442] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:21.525034 17565 raft_consensus.cc:729] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, State: Initialized, Role: FOLLOWER
I20260928 14:50:21.525028 17568 raft_consensus.cc:729] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 376cfe628b4343829c7fe5d4f29e756c, State: Initialized, Role: FOLLOWER
I20260928 14:50:21.525238 17568 consensus_queue.cc:260] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:21.525242 17565 consensus_queue.cc:260] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:21.525616 17571 tablet_replica.cc:439] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } }
I20260928 14:50:21.525784 17568 ts_tablet_manager.cc:1428] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:50:21.525871 17571 tablet_replica.cc:442] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:21.525741 17573 tablet_replica.cc:439] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } }
I20260928 14:50:21.525998 17573 tablet_replica.cc:442] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:21.526747 17565 ts_tablet_manager.cc:1428] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:50:21.833359 17572 raft_consensus.cc:482] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:21.833593 17572 raft_consensus.cc:504] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:21.835145 17572 leader_election.cc:288] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194:33021), 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457)
I20260928 14:50:21.843106 17560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.835587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:56011 (local address 127.14.241.195:35457)
0928 14:50:21.835932 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.835939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.836833 (+ 894us) server_negotiation.cc:407] Connection header received
0928 14:50:21.836990 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.836993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.837048 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.837116 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:21.837597 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.838539 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.840884 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.842028 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.842183 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.842259 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.842696 (+ 437us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.842828 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203}
I20260928 14:50:21.843106 17561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.835534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:41855 (local address 127.14.241.194:33021)
0928 14:50:21.835786 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.835793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.836260 (+ 467us) server_negotiation.cc:407] Connection header received
0928 14:50:21.836458 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.836465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.836546 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.836648 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:21.837264 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.838276 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.840910 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.842120 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.842257 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.842320 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.842695 (+ 375us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.842850 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:50:21.843327 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.835541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:35457 (local address 127.14.241.193:56011)
0928 14:50:21.836729 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.836746 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.836877 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.837157 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.837161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.837186 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:21.837447 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.837455 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.838660 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.838665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.840764 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.840773 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.842270 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.842276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.842472 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.842572 (+ 100us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.842666 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.842856 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"mutex_wait_us":95,"thread_start_us":945,"threads_started":1}
I20260928 14:50:21.843354 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.835395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:33021 (local address 127.14.241.193:41855)
0928 14:50:21.836132 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.836148 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.836310 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.836691 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.836697 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.836728 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:21.837094 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.837106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.838443 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.838448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.840728 (+ 2280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.840746 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.842311 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.842317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.842543 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.842575 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.842666 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.842854 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":77,"spinlock_wait_cycles":46336,"thread_start_us":446,"threads_started":1}
I20260928 14:50:21.844085 17357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714b63b0e7e14b21aa25ee2ece50ddcc" candidate_uuid: "882926d3969841bf8e0cb9fcb4172e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "376cfe628b4343829c7fe5d4f29e756c" is_pre_election: true
I20260928 14:50:21.844102 17496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714b63b0e7e14b21aa25ee2ece50ddcc" candidate_uuid: "882926d3969841bf8e0cb9fcb4172e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" is_pre_election: true
I20260928 14:50:21.844377 17496 raft_consensus.cc:2454] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 882926d3969841bf8e0cb9fcb4172e30 in term 0.
I20260928 14:50:21.844388 17357 raft_consensus.cc:2454] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 882926d3969841bf8e0cb9fcb4172e30 in term 0.
I20260928 14:50:21.844885 17151 leader_election.cc:302] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 376cfe628b4343829c7fe5d4f29e756c, 882926d3969841bf8e0cb9fcb4172e30; no voters:
I20260928 14:50:21.845315 17572 raft_consensus.cc:2781] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:50:21.845436 17572 raft_consensus.cc:482] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:50:21.845535 17572 raft_consensus.cc:3037] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:21.847625 17572 raft_consensus.cc:504] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:21.848183 17572 leader_election.cc:288] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [CANDIDATE]: Term 1 election: Requested vote from peers 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194:33021), 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457)
I20260928 14:50:21.848688 17357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714b63b0e7e14b21aa25ee2ece50ddcc" candidate_uuid: "882926d3969841bf8e0cb9fcb4172e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "376cfe628b4343829c7fe5d4f29e756c"
I20260928 14:50:21.848706 17496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714b63b0e7e14b21aa25ee2ece50ddcc" candidate_uuid: "882926d3969841bf8e0cb9fcb4172e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5"
I20260928 14:50:21.848928 17357 raft_consensus.cc:3037] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:21.848932 17496 raft_consensus.cc:3037] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:21.851778 17357 raft_consensus.cc:2454] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 882926d3969841bf8e0cb9fcb4172e30 in term 1.
I20260928 14:50:21.851778 17496 raft_consensus.cc:2454] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 882926d3969841bf8e0cb9fcb4172e30 in term 1.
I20260928 14:50:21.852223 17151 leader_election.cc:302] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, 376cfe628b4343829c7fe5d4f29e756c, 882926d3969841bf8e0cb9fcb4172e30; no voters:
I20260928 14:50:21.852604 17572 raft_consensus.cc:2781] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:21.853132 17572 raft_consensus.cc:686] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [term 1 LEADER]: Becoming Leader. State: Replica: 882926d3969841bf8e0cb9fcb4172e30, State: Running, Role: LEADER
I20260928 14:50:21.853653 17572 consensus_queue.cc:237] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:21.856928 17591 tablet_replica.cc:439] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: TxnStatusTablet state changed. Reason: New leader 882926d3969841bf8e0cb9fcb4172e30. Latest consensus state: current_term: 1 leader_uuid: "882926d3969841bf8e0cb9fcb4172e30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } }
I20260928 14:50:21.857324 17591 tablet_replica.cc:442] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:50:21.858017 17593 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 14:50:21.858331 17593 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 14:50:21.858454 17061 catalog_manager.cc:5495] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193). New cstate: current_term: 1 leader_uuid: "882926d3969841bf8e0cb9fcb4172e30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } health_report { overall_health: UNKNOWN } } }
I20260928 14:50:21.891811 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.885542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:40707 (local address 127.0.0.1:39254)
0928 14:50:21.885746 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.885766 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.885902 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.886457 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.886461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.886472 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:50:21.886627 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.886634 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.888024 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.888028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.889094 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.889108 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.890068 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.890072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.890222 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.890280 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:50:21.891327 (+ 1047us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:50:21.891349 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.891432 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.891562 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 14:50:21.891842 17577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.885839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39254 (local address 127.14.241.193:40707)
0928 14:50:21.886046 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.886052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.886071 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:50:21.886171 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.886177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.886283 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.886443 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:21.886756 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.887875 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.889230 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.889878 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.890013 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.890486 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:21.891197 (+ 711us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:21.891369 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.891470 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.891563 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:50:21.899376 17496 raft_consensus.cc:1261] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Refusing update from remote peer 882926d3969841bf8e0cb9fcb4172e30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:21.899696 17357 raft_consensus.cc:1261] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Refusing update from remote peer 882926d3969841bf8e0cb9fcb4172e30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:21.900152 17591 consensus_queue.cc:1035] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:21.900506 17572 consensus_queue.cc:1035] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:21.912652 17573 tablet_replica.cc:439] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c: TxnStatusTablet state changed. Reason: New leader 882926d3969841bf8e0cb9fcb4172e30. Latest consensus state: current_term: 1 leader_uuid: "882926d3969841bf8e0cb9fcb4172e30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } }
I20260928 14:50:21.912990 17573 tablet_replica.cc:442] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:21.916191 17596 mvcc.cc:205] Tried to move back new op lower bound from 7334326361691074560 to 7334326361516347392. Current Snapshot: MvccSnapshot[applied={T|T < 7334326361691074560}]
I20260928 14:50:21.919865 17571 tablet_replica.cc:439] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: TxnStatusTablet state changed. Reason: New leader 882926d3969841bf8e0cb9fcb4172e30. Latest consensus state: current_term: 1 leader_uuid: "882926d3969841bf8e0cb9fcb4172e30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } }
I20260928 14:50:21.920140 17571 tablet_replica.cc:442] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:21.923652 17573 tablet_replica.cc:439] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "882926d3969841bf8e0cb9fcb4172e30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } }
I20260928 14:50:21.923936 17573 tablet_replica.cc:442] T 714b63b0e7e14b21aa25ee2ece50ddcc P 376cfe628b4343829c7fe5d4f29e756c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:21.926678 17571 tablet_replica.cc:439] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "882926d3969841bf8e0cb9fcb4172e30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } }
I20260928 14:50:21.927008 17571 tablet_replica.cc:442] T 714b63b0e7e14b21aa25ee2ece50ddcc P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:50:21.936942 17572 tablet_replica.cc:439] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "882926d3969841bf8e0cb9fcb4172e30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } }
I20260928 14:50:21.937252 17572 tablet_replica.cc:442] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:50:21.937990 17591 tablet_replica.cc:439] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "882926d3969841bf8e0cb9fcb4172e30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } } }
I20260928 14:50:21.938236 17591 tablet_replica.cc:442] T 714b63b0e7e14b21aa25ee2ece50ddcc P 882926d3969841bf8e0cb9fcb4172e30: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:50:21.941543 17560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.934901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.14.241.195:35457)
0928 14:50:21.935076 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.935082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.935244 (+ 162us) server_negotiation.cc:407] Connection header received
0928 14:50:21.935567 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.935573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.935690 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.935844 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:21.936316 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.937428 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.939066 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.939595 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.939690 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.940342 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:21.940958 (+ 616us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:21.941085 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.941188 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.941272 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:50:21.957623 17577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.951863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:44897 (local address 127.14.241.193:40707)
0928 14:50:21.952024 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:21.952028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:21.952791 (+ 763us) server_negotiation.cc:407] Connection header received
0928 14:50:21.952965 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:21.952969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:21.953062 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:21.953143 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:21.953558 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.954473 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.955770 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.956243 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.956386 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.956858 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:21.956986 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:21.957061 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:21.957203 (+ 142us) server_negotiation.cc:299] Negotiation successful
0928 14:50:21.957283 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:50:21.957721 17609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:21.951768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:40707 (local address 127.14.241.195:44897)
0928 14:50:21.952687 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:21.952703 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:21.952833 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:21.953162 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:21.953166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:21.953178 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:50:21.953423 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.953433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.954591 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.954596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.955639 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:21.955650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:21.956453 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:21.956460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:21.956598 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:21.956651 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:50:21.957075 (+ 424us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:50:21.957100 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 14:50:21.957167 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 14:50:21.957298 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"thread_start_us":659,"threads_started":1}
I20260928 14:50:21.972707 17218 raft_consensus.cc:1261] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Refusing update from remote peer 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:21.972728 17357 raft_consensus.cc:1261] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Refusing update from remote peer 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:21.973757 17571 consensus_queue.cc:1035] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:21.974920 17571 consensus_queue.cc:1035] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:21.988451 17597 mvcc.cc:205] Tried to move back new op lower bound from 7334326361991933952 to 7334326359729811456. Current Snapshot: MvccSnapshot[applied={T|T < 7334326361991933952}]
I20260928 14:50:22.007486 17599 mvcc.cc:205] Tried to move back new op lower bound from 7334326361991933952 to 7334326359729811456. Current Snapshot: MvccSnapshot[applied={T|T < 7334326361991933952}]
I20260928 14:50:22.040225 17617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:22.018207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:35457 (local address 127.14.241.193:46345)
0928 14:50:22.033611 (+ 15404us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:22.033628 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:22.033787 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:22.034172 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:22.034177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:22.034191 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:50:22.034501 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:22.034515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:22.035950 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:22.035956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:22.037716 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:22.037732 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:22.038808 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:22.038814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:22.039007 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:22.039091 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:50:22.039564 (+ 473us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:50:22.039575 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 14:50:22.039651 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:50:22.039801 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15166,"thread_start_us":15034,"threads_started":1}
I20260928 14:50:22.041059 17560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:22.018627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:46345 (local address 127.14.241.195:35457)
0928 14:50:22.018822 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:22.018828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:22.033735 (+ 14907us) server_negotiation.cc:407] Connection header received
0928 14:50:22.033928 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:22.033933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:22.034020 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:22.034110 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:22.034674 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:22.035768 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:22.037903 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:22.038559 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:22.038678 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:22.039304 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:22.039441 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:22.039516 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:22.040671 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0928 14:50:22.040762 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:50:22.082746 17061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:22.083079 17061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:22.087489 17061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f: Sending DeleteTablet for 3 replicas of tablet 3d75818c49a4472d8b792d2b0edc43ee
I20260928 14:50:22.088825 17337 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d75818c49a4472d8b792d2b0edc43ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:22 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59040
I20260928 14:50:22.088842 17476 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d75818c49a4472d8b792d2b0edc43ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:22 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59142
I20260928 14:50:22.091789 17059 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:50:22.092622 17059 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:22.096076 17198 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d75818c49a4472d8b792d2b0edc43ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:22 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:39244
I20260928 14:50:22.101694 17196 tablet_service.cc:1467] Processing CreateTablet for tablet deded01a4c9e43b3b8a45991d068ce65 (DEFAULT_TABLE table=test-table [id=08fcf8bf7ce74f30b7e02c8959d0505e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:22.101976 17475 tablet_service.cc:1467] Processing CreateTablet for tablet deded01a4c9e43b3b8a45991d068ce65 (DEFAULT_TABLE table=test-table [id=08fcf8bf7ce74f30b7e02c8959d0505e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:22.102298 17196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deded01a4c9e43b3b8a45991d068ce65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:22.102560 17475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deded01a4c9e43b3b8a45991d068ce65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:22.104914 17336 tablet_service.cc:1467] Processing CreateTablet for tablet deded01a4c9e43b3b8a45991d068ce65 (DEFAULT_TABLE table=test-table [id=08fcf8bf7ce74f30b7e02c8959d0505e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:22.105454 17336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deded01a4c9e43b3b8a45991d068ce65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:22.113454 17621 tablet_replica.cc:330] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: stopping tablet replica
I20260928 14:50:22.114035 17621 raft_consensus.cc:2229] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:50:22.115921 17621 raft_consensus.cc:2258] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:22.116264 17623 tablet_replica.cc:330] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: stopping tablet replica
I20260928 14:50:22.116747 17623 raft_consensus.cc:2229] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:22.117115 17623 raft_consensus.cc:2258] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:22.118103 17621 ts_tablet_manager.cc:1905] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:22.118793 17626 tablet_bootstrap.cc:492] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c: Bootstrap starting.
I20260928 14:50:22.119720 17622 tablet_replica.cc:330] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: stopping tablet replica
I20260928 14:50:22.120189 17622 raft_consensus.cc:2229] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:22.120594 17622 raft_consensus.cc:2258] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:22.120774 17626 tablet_bootstrap.cc:654] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:22.122685 17623 ts_tablet_manager.cc:1905] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:22.123521 17625 tablet_bootstrap.cc:492] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Bootstrap starting.
I20260928 14:50:22.125605 17625 tablet_bootstrap.cc:654] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:22.126524 17622 ts_tablet_manager.cc:1905] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:22.127694 17624 tablet_bootstrap.cc:492] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30: Bootstrap starting.
I20260928 14:50:22.129387 17621 ts_tablet_manager.cc:1918] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:50:22.129534 17624 tablet_bootstrap.cc:654] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:22.129673 17621 log.cc:1196] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/3d75818c49a4472d8b792d2b0edc43ee
I20260928 14:50:22.131644 17621 ts_tablet_manager.cc:1939] T 3d75818c49a4472d8b792d2b0edc43ee P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting consensus metadata
I20260928 14:50:22.133594 17626 tablet_bootstrap.cc:492] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c: No bootstrap required, opened a new log
I20260928 14:50:22.133562 17045 catalog_manager.cc:4841] TS 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457): tablet 3d75818c49a4472d8b792d2b0edc43ee (table test-table [id=37ab10d2c1bc4dd2a902a6272f4fb885]) successfully deleted
I20260928 14:50:22.133738 17626 ts_tablet_manager.cc:1397] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260928 14:50:22.134531 17626 raft_consensus.cc:348] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:22.134763 17626 raft_consensus.cc:374] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:22.134845 17626 raft_consensus.cc:729] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 376cfe628b4343829c7fe5d4f29e756c, State: Initialized, Role: FOLLOWER
I20260928 14:50:22.137559 17623 ts_tablet_manager.cc:1918] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:50:22.137741 17623 log.cc:1196] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/3d75818c49a4472d8b792d2b0edc43ee
I20260928 14:50:22.138270 17623 ts_tablet_manager.cc:1939] T 3d75818c49a4472d8b792d2b0edc43ee P 882926d3969841bf8e0cb9fcb4172e30: Deleting consensus metadata
I20260928 14:50:22.137028 17626 consensus_queue.cc:260] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:22.140098 17048 catalog_manager.cc:4841] TS 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707): tablet 3d75818c49a4472d8b792d2b0edc43ee (table test-table [id=37ab10d2c1bc4dd2a902a6272f4fb885]) successfully deleted
I20260928 14:50:22.140738 17626 ts_tablet_manager.cc:1428] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260928 14:50:22.140883 17624 tablet_bootstrap.cc:492] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30: No bootstrap required, opened a new log
I20260928 14:50:22.141085 17624 ts_tablet_manager.cc:1397] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260928 14:50:22.142007 17622 ts_tablet_manager.cc:1918] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:50:22.142182 17622 log.cc:1196] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/3d75818c49a4472d8b792d2b0edc43ee
I20260928 14:50:22.142221 17624 raft_consensus.cc:348] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:22.142611 17624 raft_consensus.cc:374] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:22.142707 17622 ts_tablet_manager.cc:1939] T 3d75818c49a4472d8b792d2b0edc43ee P 376cfe628b4343829c7fe5d4f29e756c: Deleting consensus metadata
I20260928 14:50:22.142824 17624 raft_consensus.cc:729] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 882926d3969841bf8e0cb9fcb4172e30, State: Initialized, Role: FOLLOWER
I20260928 14:50:22.143108 17624 consensus_queue.cc:260] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:22.143855 17624 ts_tablet_manager.cc:1428] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:50:22.146624 17048 catalog_manager.cc:4841] TS 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194:33021): tablet 3d75818c49a4472d8b792d2b0edc43ee (table test-table [id=37ab10d2c1bc4dd2a902a6272f4fb885]) successfully deleted
I20260928 14:50:22.146845 17625 tablet_bootstrap.cc:492] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: No bootstrap required, opened a new log
I20260928 14:50:22.147053 17625 ts_tablet_manager.cc:1397] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.003s
I20260928 14:50:22.147933 17625 raft_consensus.cc:348] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:22.148257 17625 raft_consensus.cc:374] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:22.148488 17625 raft_consensus.cc:729] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, State: Initialized, Role: FOLLOWER
I20260928 14:50:22.148769 17625 consensus_queue.cc:260] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:22.149428 17625 ts_tablet_manager.cc:1428] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:50:22.180895 17604 raft_consensus.cc:482] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:22.181088 17604 raft_consensus.cc:504] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:22.181836 17604 leader_election.cc:288] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194:33021), 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457)
I20260928 14:50:22.183177 17356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deded01a4c9e43b3b8a45991d068ce65" candidate_uuid: "882926d3969841bf8e0cb9fcb4172e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "376cfe628b4343829c7fe5d4f29e756c" is_pre_election: true
I20260928 14:50:22.183459 17356 raft_consensus.cc:2454] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 882926d3969841bf8e0cb9fcb4172e30 in term 0.
I20260928 14:50:22.183873 17495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deded01a4c9e43b3b8a45991d068ce65" candidate_uuid: "882926d3969841bf8e0cb9fcb4172e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" is_pre_election: true
I20260928 14:50:22.184178 17495 raft_consensus.cc:2454] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 882926d3969841bf8e0cb9fcb4172e30 in term 0.
I20260928 14:50:22.184904 17148 leader_election.cc:302] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, 882926d3969841bf8e0cb9fcb4172e30; no voters:
I20260928 14:50:22.185670 17604 raft_consensus.cc:2781] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:50:22.185782 17604 raft_consensus.cc:482] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:50:22.185860 17604 raft_consensus.cc:3037] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:22.189226 17604 raft_consensus.cc:504] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:22.190301 17356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deded01a4c9e43b3b8a45991d068ce65" candidate_uuid: "882926d3969841bf8e0cb9fcb4172e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "376cfe628b4343829c7fe5d4f29e756c"
I20260928 14:50:22.190531 17356 raft_consensus.cc:3037] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:22.193660 17356 raft_consensus.cc:2454] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 882926d3969841bf8e0cb9fcb4172e30 in term 1.
I20260928 14:50:22.194345 17151 leader_election.cc:302] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 376cfe628b4343829c7fe5d4f29e756c, 882926d3969841bf8e0cb9fcb4172e30; no voters:
I20260928 14:50:22.194669 17591 raft_consensus.cc:2781] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:22.194808 17591 raft_consensus.cc:686] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 LEADER]: Becoming Leader. State: Replica: 882926d3969841bf8e0cb9fcb4172e30, State: Running, Role: LEADER
I20260928 14:50:22.195065 17591 consensus_queue.cc:237] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:22.198091 17495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deded01a4c9e43b3b8a45991d068ce65" candidate_uuid: "882926d3969841bf8e0cb9fcb4172e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5"
I20260928 14:50:22.198305 17495 raft_consensus.cc:3037] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:22.201649 17495 raft_consensus.cc:2454] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 882926d3969841bf8e0cb9fcb4172e30 in term 1.
I20260928 14:50:22.202472 17604 leader_election.cc:288] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [CANDIDATE]: Term 1 election: Requested vote from peers 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194:33021), 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457)
I20260928 14:50:22.211949 17059 catalog_manager.cc:5495] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193). New cstate: current_term: 1 leader_uuid: "882926d3969841bf8e0cb9fcb4172e30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } health_report { overall_health: UNKNOWN } } }
I20260928 14:50:22.256755 17577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:22.251480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39268 (local address 127.14.241.193:40707)
0928 14:50:22.251625 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:22.251631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:22.251648 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:50:22.251720 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:22.251724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:22.251809 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:22.251889 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:22.252504 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:22.253612 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:22.255018 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:22.255487 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:22.255590 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:22.255957 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:22.256083 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:22.256155 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:22.256343 (+ 188us) server_negotiation.cc:299] Negotiation successful
0928 14:50:22.256423 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:50:22.270416 17617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:22.265377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:40707 (local address 127.14.241.193:52321)
0928 14:50:22.265605 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:22.265629 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:22.265785 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:22.266123 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:22.266127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:22.266137 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:50:22.266323 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:22.266332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:22.267329 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:22.267335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:22.268454 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:22.268467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:22.269216 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:22.269231 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:22.269361 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:22.269396 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:50:22.269783 (+ 387us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:50:22.269794 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 14:50:22.269874 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 14:50:22.270064 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":18}
I20260928 14:50:22.270519 17577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:22.265411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:52321 (local address 127.14.241.193:40707)
0928 14:50:22.265565 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:22.265569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:22.265738 (+ 169us) server_negotiation.cc:407] Connection header received
0928 14:50:22.265911 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:22.265915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:22.265996 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:22.266097 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:22.266440 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:22.267186 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:22.268582 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:22.269063 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:22.269175 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:22.269577 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:22.269692 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:22.269771 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:22.269895 (+ 124us) server_negotiation.cc:299] Negotiation successful
0928 14:50:22.270023 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":48}
I20260928 14:50:22.279382 17355 raft_consensus.cc:1261] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Refusing update from remote peer 882926d3969841bf8e0cb9fcb4172e30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:22.280660 17496 raft_consensus.cc:1261] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Refusing update from remote peer 882926d3969841bf8e0cb9fcb4172e30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:22.281013 17572 consensus_queue.cc:1035] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:22.281862 17610 consensus_queue.cc:1035] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 14:50:22.312575 17634 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 14:50:23.915952 17059 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:23.916221 17059 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:23.920451 17059 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f: Sending DeleteTablet for 3 replicas of tablet deded01a4c9e43b3b8a45991d068ce65
I20260928 14:50:23.921361 17475 tablet_service.cc:1514] Processing DeleteTablet for tablet deded01a4c9e43b3b8a45991d068ce65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:23 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59142
I20260928 14:50:23.921387 17196 tablet_service.cc:1514] Processing DeleteTablet for tablet deded01a4c9e43b3b8a45991d068ce65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:23 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:39244
I20260928 14:50:23.921582 17337 tablet_service.cc:1514] Processing DeleteTablet for tablet deded01a4c9e43b3b8a45991d068ce65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:23 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59040
I20260928 14:50:23.932366 17656 tablet_replica.cc:330] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c: stopping tablet replica
I20260928 14:50:23.932602 17656 raft_consensus.cc:2229] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:23.932860 17656 raft_consensus.cc:2258] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:23.933410 17655 tablet_replica.cc:330] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30: stopping tablet replica
I20260928 14:50:23.933611 17655 raft_consensus.cc:2229] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:50:23.933779 17654 tablet_replica.cc:330] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: stopping tablet replica
I20260928 14:50:23.933965 17654 raft_consensus.cc:2229] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:23.934190 17654 raft_consensus.cc:2258] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:23.934199 17655 raft_consensus.cc:2258] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:23.934423 17656 ts_tablet_manager.cc:1905] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:23.935166 17654 ts_tablet_manager.cc:1905] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:23.936916 17655 ts_tablet_manager.cc:1905] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:23.940214 17656 ts_tablet_manager.cc:1918] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:50:23.940389 17656 log.cc:1196] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/deded01a4c9e43b3b8a45991d068ce65
I20260928 14:50:23.940830 17656 ts_tablet_manager.cc:1939] T deded01a4c9e43b3b8a45991d068ce65 P 376cfe628b4343829c7fe5d4f29e756c: Deleting consensus metadata
I20260928 14:50:23.941256 17654 ts_tablet_manager.cc:1918] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:50:23.941391 17654 log.cc:1196] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/deded01a4c9e43b3b8a45991d068ce65
I20260928 14:50:23.941810 17654 ts_tablet_manager.cc:1939] T deded01a4c9e43b3b8a45991d068ce65 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting consensus metadata
I20260928 14:50:23.942075 17048 catalog_manager.cc:4841] TS 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194:33021): tablet deded01a4c9e43b3b8a45991d068ce65 (table test-table [id=08fcf8bf7ce74f30b7e02c8959d0505e]) successfully deleted
I20260928 14:50:23.942888 17045 catalog_manager.cc:4841] TS 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457): tablet deded01a4c9e43b3b8a45991d068ce65 (table test-table [id=08fcf8bf7ce74f30b7e02c8959d0505e]) successfully deleted
I20260928 14:50:23.943710 17655 ts_tablet_manager.cc:1918] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:50:23.943837 17655 log.cc:1196] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/deded01a4c9e43b3b8a45991d068ce65
I20260928 14:50:23.944319 17655 ts_tablet_manager.cc:1939] T deded01a4c9e43b3b8a45991d068ce65 P 882926d3969841bf8e0cb9fcb4172e30: Deleting consensus metadata
I20260928 14:50:23.945350 17048 catalog_manager.cc:4841] TS 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707): tablet deded01a4c9e43b3b8a45991d068ce65 (table test-table [id=08fcf8bf7ce74f30b7e02c8959d0505e]) successfully deleted
Sep 28 14:50:23 dist-test-slave-vk40 krb5kdc[17016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607023, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:50:23 dist-test-slave-vk40 krb5kdc[17016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607023, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:23.981057 17664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:23.963327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57126 (local address 127.14.241.254:43897)
0928 14:50:23.963812 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:23.963838 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:23.963863 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:50:23.964019 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:23.964023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:23.964094 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:23.964194 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:23.965838 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:23.966603 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:23.967837 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:23.968253 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:23.968398 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:23.975660 (+ 7262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:23.975666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:23.975670 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:23.975711 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:23.978580 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:23.979291 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:23.979296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:23.979298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:23.979430 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:23.979712 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:23.979715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:23.979717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:23.979999 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:23.980177 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:23.980503 (+ 326us) server_negotiation.cc:299] Negotiation successful
0928 14:50:23.980728 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":246,"threads_started":1}
I20260928 14:50:23.984599 17059 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:50:23.985167 17059 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:23.991400 17196 tablet_service.cc:1467] Processing CreateTablet for tablet a8c22afe8da44f7d832783080670b942 (DEFAULT_TABLE table=test-table [id=ba49a85f1a5240c999693c80fae7ce24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:23.991582 17337 tablet_service.cc:1467] Processing CreateTablet for tablet a8c22afe8da44f7d832783080670b942 (DEFAULT_TABLE table=test-table [id=ba49a85f1a5240c999693c80fae7ce24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:23.991753 17475 tablet_service.cc:1467] Processing CreateTablet for tablet a8c22afe8da44f7d832783080670b942 (DEFAULT_TABLE table=test-table [id=ba49a85f1a5240c999693c80fae7ce24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:23.991904 17196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c22afe8da44f7d832783080670b942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:23.991994 17337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c22afe8da44f7d832783080670b942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:23.992158 17475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c22afe8da44f7d832783080670b942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:23.999230 17666 tablet_bootstrap.cc:492] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Bootstrap starting.
I20260928 14:50:24.000021 17665 tablet_bootstrap.cc:492] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30: Bootstrap starting.
I20260928 14:50:24.000628 17667 tablet_bootstrap.cc:492] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c: Bootstrap starting.
I20260928 14:50:24.001474 17666 tablet_bootstrap.cc:654] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:24.001691 17665 tablet_bootstrap.cc:654] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:24.002205 17667 tablet_bootstrap.cc:654] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:24.003661 17665 tablet_bootstrap.cc:492] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30: No bootstrap required, opened a new log
I20260928 14:50:24.003796 17665 ts_tablet_manager.cc:1397] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260928 14:50:24.004011 17667 tablet_bootstrap.cc:492] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c: No bootstrap required, opened a new log
I20260928 14:50:24.004134 17667 ts_tablet_manager.cc:1397] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260928 14:50:24.004583 17665 raft_consensus.cc:348] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.004778 17665 raft_consensus.cc:374] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:24.004873 17665 raft_consensus.cc:729] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 882926d3969841bf8e0cb9fcb4172e30, State: Initialized, Role: FOLLOWER
I20260928 14:50:24.004863 17667 raft_consensus.cc:348] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.005105 17667 raft_consensus.cc:374] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:24.005131 17666 tablet_bootstrap.cc:492] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: No bootstrap required, opened a new log
I20260928 14:50:24.005204 17667 raft_consensus.cc:729] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 376cfe628b4343829c7fe5d4f29e756c, State: Initialized, Role: FOLLOWER
I20260928 14:50:24.005110 17665 consensus_queue.cc:260] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.005280 17666 ts_tablet_manager.cc:1397] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260928 14:50:24.005465 17667 consensus_queue.cc:260] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.005657 17665 ts_tablet_manager.cc:1428] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:50:24.006049 17666 raft_consensus.cc:348] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.006273 17666 raft_consensus.cc:374] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:24.006359 17666 raft_consensus.cc:729] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, State: Initialized, Role: FOLLOWER
I20260928 14:50:24.006595 17666 consensus_queue.cc:260] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.007455 17666 ts_tablet_manager.cc:1428] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:50:24.008422 17667 ts_tablet_manager.cc:1428] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260928 14:50:24.050760 17671 raft_consensus.cc:482] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:24.050990 17671 raft_consensus.cc:504] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.051750 17671 leader_election.cc:288] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707), 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457)
I20260928 14:50:24.052261 17218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c22afe8da44f7d832783080670b942" candidate_uuid: "376cfe628b4343829c7fe5d4f29e756c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882926d3969841bf8e0cb9fcb4172e30" is_pre_election: true
I20260928 14:50:24.052297 17496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c22afe8da44f7d832783080670b942" candidate_uuid: "376cfe628b4343829c7fe5d4f29e756c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" is_pre_election: true
I20260928 14:50:24.052520 17218 raft_consensus.cc:2454] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 376cfe628b4343829c7fe5d4f29e756c in term 0.
I20260928 14:50:24.052552 17496 raft_consensus.cc:2454] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 376cfe628b4343829c7fe5d4f29e756c in term 0.
I20260928 14:50:24.053047 17288 leader_election.cc:302] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, 376cfe628b4343829c7fe5d4f29e756c, 882926d3969841bf8e0cb9fcb4172e30; no voters:
I20260928 14:50:24.053313 17671 raft_consensus.cc:2781] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:50:24.053436 17671 raft_consensus.cc:482] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:50:24.053519 17671 raft_consensus.cc:3037] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:24.055565 17671 raft_consensus.cc:504] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.056099 17671 leader_election.cc:288] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [CANDIDATE]: Term 1 election: Requested vote from peers 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707), 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457)
I20260928 14:50:24.056641 17496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c22afe8da44f7d832783080670b942" candidate_uuid: "376cfe628b4343829c7fe5d4f29e756c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5"
I20260928 14:50:24.056656 17218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c22afe8da44f7d832783080670b942" candidate_uuid: "376cfe628b4343829c7fe5d4f29e756c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882926d3969841bf8e0cb9fcb4172e30"
I20260928 14:50:24.056870 17218 raft_consensus.cc:3037] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:24.056897 17496 raft_consensus.cc:3037] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:24.059763 17496 raft_consensus.cc:2454] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 376cfe628b4343829c7fe5d4f29e756c in term 1.
I20260928 14:50:24.059778 17218 raft_consensus.cc:2454] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 376cfe628b4343829c7fe5d4f29e756c in term 1.
I20260928 14:50:24.060189 17288 leader_election.cc:302] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, 376cfe628b4343829c7fe5d4f29e756c; no voters:
I20260928 14:50:24.060527 17671 raft_consensus.cc:2781] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:24.061076 17671 raft_consensus.cc:686] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 1 LEADER]: Becoming Leader. State: Replica: 376cfe628b4343829c7fe5d4f29e756c, State: Running, Role: LEADER
I20260928 14:50:24.061484 17671 consensus_queue.cc:237] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.065030 17061 catalog_manager.cc:5495] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c reported cstate change: term changed from 0 to 1, leader changed from <none> to 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194). New cstate: current_term: 1 leader_uuid: "376cfe628b4343829c7fe5d4f29e756c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } health_report { overall_health: UNKNOWN } } }
I20260928 14:50:24.094473 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:24.088950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.14.241.194:33021)
0928 14:50:24.089498 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:24.089502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:24.089520 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:50:24.089604 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:24.089608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:24.089685 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:24.089830 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:24.090226 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:24.090986 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:24.092268 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:24.092696 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:24.092817 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:24.093202 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:24.093834 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:24.093977 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:24.094075 (+ 98us) server_negotiation.cc:299] Negotiation successful
0928 14:50:24.094156 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":292,"threads_started":1}
I20260928 14:50:24.108928 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:24.104094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:38305 (local address 127.14.241.193:40707)
0928 14:50:24.104627 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:24.104630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:24.104650 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:50:24.104741 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:24.104745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:24.104810 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:24.104898 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:24.105293 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:24.106052 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:24.107413 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:24.107796 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:24.107889 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:24.108271 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:24.108421 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:24.108489 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:24.108611 (+ 122us) server_negotiation.cc:299] Negotiation successful
0928 14:50:24.108675 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":314,"threads_started":1}
I20260928 14:50:24.108981 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:24.103839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:40707 (local address 127.14.241.194:38305)
0928 14:50:24.104457 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:24.104473 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:24.104578 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:24.104912 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:24.104915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:24.104924 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:50:24.105165 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:24.105173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:24.106184 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:24.106188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:24.107290 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:24.107303 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:24.107936 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:24.107940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:24.108058 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:24.108114 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:50:24.108506 (+ 392us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:50:24.108515 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 14:50:24.108578 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 14:50:24.108681 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":279,"threads_started":1}
I20260928 14:50:24.117687 17218 raft_consensus.cc:1261] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Refusing update from remote peer 376cfe628b4343829c7fe5d4f29e756c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:24.117660 17496 raft_consensus.cc:1261] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Refusing update from remote peer 376cfe628b4343829c7fe5d4f29e756c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:24.118412 17673 consensus_queue.cc:1035] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:24.118793 17671 consensus_queue.cc:1035] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [LEADER]: Connected to new peer: Peer: permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:24.144253 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:24.139317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:33013 (local address 127.14.241.194:33021)
0928 14:50:24.139637 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:24.139644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:24.139888 (+ 244us) server_negotiation.cc:407] Connection header received
0928 14:50:24.140073 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:24.140078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:24.140138 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:24.140208 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:24.140680 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:24.141369 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:24.142661 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:24.143069 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:24.143171 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:24.143546 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:24.143656 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:24.143773 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:24.143899 (+ 126us) server_negotiation.cc:299] Negotiation successful
0928 14:50:24.143979 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188}
I20260928 14:50:24.144382 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:24.139244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:33021 (local address 127.14.241.193:33013)
0928 14:50:24.139791 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:24.139807 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:24.139949 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:24.140234 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:24.140238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:24.140250 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:50:24.140537 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:24.140545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:24.141486 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:24.141491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:24.142529 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:24.142541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:24.143228 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:24.143235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:24.143372 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:24.143410 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:50:24.143782 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:50:24.143794 (+ 12us) client_negotiation.cc:769] Sending connection context
0928 14:50:24.143875 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 14:50:24.144009 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":270,"threads_started":1}
I20260928 14:50:24.179040 17059 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:24.179364 17059 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:24.183264 17059 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f: Sending DeleteTablet for 3 replicas of tablet a8c22afe8da44f7d832783080670b942
I20260928 14:50:24.184430 17475 tablet_service.cc:1514] Processing DeleteTablet for tablet a8c22afe8da44f7d832783080670b942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:24 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59142
I20260928 14:50:24.184614 17337 tablet_service.cc:1514] Processing DeleteTablet for tablet a8c22afe8da44f7d832783080670b942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:24 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59040
I20260928 14:50:24.184757 17654 tablet_replica.cc:330] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: stopping tablet replica
I20260928 14:50:24.184921 17656 tablet_replica.cc:330] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c: stopping tablet replica
I20260928 14:50:24.184994 17654 raft_consensus.cc:2229] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:24.185173 17656 raft_consensus.cc:2229] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:50:24.185217 17654 raft_consensus.cc:2258] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:24.185338 17196 tablet_service.cc:1514] Processing DeleteTablet for tablet a8c22afe8da44f7d832783080670b942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:24 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:39244
I20260928 14:50:24.185671 17655 tablet_replica.cc:330] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30: stopping tablet replica
I20260928 14:50:24.185659 17656 raft_consensus.cc:2258] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:24.185979 17655 raft_consensus.cc:2229] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:24.186192 17655 raft_consensus.cc:2258] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:24.186568 17654 ts_tablet_manager.cc:1905] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:24.187454 17059 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:50:24.188208 17059 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:24.189486 17656 ts_tablet_manager.cc:1905] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:24.190491 17655 ts_tablet_manager.cc:1905] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:24.200834 17475 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb012d728f64714bf40de0c77e77331 (DEFAULT_TABLE table=test-table [id=07beed6142844436bd6f18e23c32fc4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:24.201385 17475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb012d728f64714bf40de0c77e77331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:24.204139 17196 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb012d728f64714bf40de0c77e77331 (DEFAULT_TABLE table=test-table [id=07beed6142844436bd6f18e23c32fc4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:24.204622 17196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb012d728f64714bf40de0c77e77331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:24.206604 17654 ts_tablet_manager.cc:1918] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:50:24.206969 17654 log.cc:1196] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/a8c22afe8da44f7d832783080670b942
I20260928 14:50:24.207630 17654 ts_tablet_manager.cc:1939] T a8c22afe8da44f7d832783080670b942 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting consensus metadata
I20260928 14:50:24.208773 17045 catalog_manager.cc:4841] TS 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457): tablet a8c22afe8da44f7d832783080670b942 (table test-table [id=ba49a85f1a5240c999693c80fae7ce24]) successfully deleted
I20260928 14:50:24.209183 17337 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb012d728f64714bf40de0c77e77331 (DEFAULT_TABLE table=test-table [id=07beed6142844436bd6f18e23c32fc4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:24.209702 17666 tablet_bootstrap.cc:492] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Bootstrap starting.
I20260928 14:50:24.209928 17337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb012d728f64714bf40de0c77e77331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:24.212929 17666 tablet_bootstrap.cc:654] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:24.214668 17655 ts_tablet_manager.cc:1918] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:50:24.214862 17655 log.cc:1196] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/a8c22afe8da44f7d832783080670b942
I20260928 14:50:24.215365 17655 ts_tablet_manager.cc:1939] T a8c22afe8da44f7d832783080670b942 P 882926d3969841bf8e0cb9fcb4172e30: Deleting consensus metadata
I20260928 14:50:24.215747 17665 tablet_bootstrap.cc:492] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30: Bootstrap starting.
I20260928 14:50:24.216630 17048 catalog_manager.cc:4841] TS 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707): tablet a8c22afe8da44f7d832783080670b942 (table test-table [id=ba49a85f1a5240c999693c80fae7ce24]) successfully deleted
I20260928 14:50:24.217796 17665 tablet_bootstrap.cc:654] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:24.226470 17665 tablet_bootstrap.cc:492] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30: No bootstrap required, opened a new log
I20260928 14:50:24.226750 17665 ts_tablet_manager.cc:1397] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260928 14:50:24.228123 17665 raft_consensus.cc:348] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.228469 17665 raft_consensus.cc:374] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:24.228583 17665 raft_consensus.cc:729] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 882926d3969841bf8e0cb9fcb4172e30, State: Initialized, Role: FOLLOWER
I20260928 14:50:24.228981 17665 consensus_queue.cc:260] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.229682 17665 ts_tablet_manager.cc:1428] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:24.233821 17667 tablet_bootstrap.cc:492] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c: Bootstrap starting.
I20260928 14:50:24.236608 17667 tablet_bootstrap.cc:654] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:24.238209 17656 ts_tablet_manager.cc:1918] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:50:24.238576 17656 log.cc:1196] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/a8c22afe8da44f7d832783080670b942
I20260928 14:50:24.240252 17656 ts_tablet_manager.cc:1939] T a8c22afe8da44f7d832783080670b942 P 376cfe628b4343829c7fe5d4f29e756c: Deleting consensus metadata
I20260928 14:50:24.240639 17666 tablet_bootstrap.cc:492] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: No bootstrap required, opened a new log
I20260928 14:50:24.241005 17666 ts_tablet_manager.cc:1397] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Time spent bootstrapping tablet: real 0.031s user 0.017s sys 0.005s
I20260928 14:50:24.241636 17048 catalog_manager.cc:4841] TS 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194:33021): tablet a8c22afe8da44f7d832783080670b942 (table test-table [id=ba49a85f1a5240c999693c80fae7ce24]) successfully deleted
I20260928 14:50:24.242481 17666 raft_consensus.cc:348] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.242920 17666 raft_consensus.cc:374] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:24.243135 17666 raft_consensus.cc:729] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, State: Initialized, Role: FOLLOWER
I20260928 14:50:24.243621 17667 tablet_bootstrap.cc:492] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c: No bootstrap required, opened a new log
I20260928 14:50:24.243546 17666 consensus_queue.cc:260] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.243978 17667 ts_tablet_manager.cc:1397] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260928 14:50:24.244395 17666 ts_tablet_manager.cc:1428] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:24.244779 17667 raft_consensus.cc:348] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.245024 17667 raft_consensus.cc:374] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:24.245116 17667 raft_consensus.cc:729] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 376cfe628b4343829c7fe5d4f29e756c, State: Initialized, Role: FOLLOWER
I20260928 14:50:24.245369 17667 consensus_queue.cc:260] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.246045 17667 ts_tablet_manager.cc:1428] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:50:24.450286 17673 raft_consensus.cc:482] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:24.450528 17673 raft_consensus.cc:504] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.451239 17673 leader_election.cc:288] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707), 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457)
I20260928 14:50:24.451922 17218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb012d728f64714bf40de0c77e77331" candidate_uuid: "376cfe628b4343829c7fe5d4f29e756c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882926d3969841bf8e0cb9fcb4172e30" is_pre_election: true
I20260928 14:50:24.452099 17495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb012d728f64714bf40de0c77e77331" candidate_uuid: "376cfe628b4343829c7fe5d4f29e756c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" is_pre_election: true
I20260928 14:50:24.452337 17218 raft_consensus.cc:2454] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 376cfe628b4343829c7fe5d4f29e756c in term 0.
I20260928 14:50:24.452382 17495 raft_consensus.cc:2454] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 376cfe628b4343829c7fe5d4f29e756c in term 0.
I20260928 14:50:24.452821 17288 leader_election.cc:302] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5, 376cfe628b4343829c7fe5d4f29e756c; no voters:
I20260928 14:50:24.453239 17673 raft_consensus.cc:2781] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:50:24.453346 17673 raft_consensus.cc:482] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:50:24.453469 17673 raft_consensus.cc:3037] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:24.455628 17673 raft_consensus.cc:504] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.456387 17673 leader_election.cc:288] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [CANDIDATE]: Term 1 election: Requested vote from peers 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707), 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457)
I20260928 14:50:24.457072 17218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb012d728f64714bf40de0c77e77331" candidate_uuid: "376cfe628b4343829c7fe5d4f29e756c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882926d3969841bf8e0cb9fcb4172e30"
I20260928 14:50:24.457082 17495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb012d728f64714bf40de0c77e77331" candidate_uuid: "376cfe628b4343829c7fe5d4f29e756c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5"
I20260928 14:50:24.457316 17218 raft_consensus.cc:3037] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:24.457461 17495 raft_consensus.cc:3037] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:24.460448 17218 raft_consensus.cc:2454] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 376cfe628b4343829c7fe5d4f29e756c in term 1.
I20260928 14:50:24.460811 17495 raft_consensus.cc:2454] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 376cfe628b4343829c7fe5d4f29e756c in term 1.
I20260928 14:50:24.460990 17291 leader_election.cc:302] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 376cfe628b4343829c7fe5d4f29e756c, 882926d3969841bf8e0cb9fcb4172e30; no voters:
I20260928 14:50:24.461329 17673 raft_consensus.cc:2781] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:24.461634 17673 raft_consensus.cc:686] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 1 LEADER]: Becoming Leader. State: Replica: 376cfe628b4343829c7fe5d4f29e756c, State: Running, Role: LEADER
I20260928 14:50:24.461953 17673 consensus_queue.cc:237] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } }
I20260928 14:50:24.465512 17061 catalog_manager.cc:5495] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c reported cstate change: term changed from 0 to 1, leader changed from <none> to 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194). New cstate: current_term: 1 leader_uuid: "376cfe628b4343829c7fe5d4f29e756c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "376cfe628b4343829c7fe5d4f29e756c" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 33021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 } health_report { overall_health: UNKNOWN } } }
I20260928 14:50:24.525664 17495 raft_consensus.cc:1261] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Refusing update from remote peer 376cfe628b4343829c7fe5d4f29e756c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:24.525676 17218 raft_consensus.cc:1261] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Refusing update from remote peer 376cfe628b4343829c7fe5d4f29e756c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:24.526324 17673 consensus_queue.cc:1035] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:24.526715 17683 consensus_queue.cc:1035] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [LEADER]: Connected to new peer: Peer: permanent_uuid: "882926d3969841bf8e0cb9fcb4172e30" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 14:50:24.568840 17639 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 14:50:26.169009 17059 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:26.169281 17059 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:26.174659 17059 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d56291ab4d54a3182810ea9f337785f: Sending DeleteTablet for 3 replicas of tablet 9cb012d728f64714bf40de0c77e77331
I20260928 14:50:26.175448 17196 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb012d728f64714bf40de0c77e77331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:26 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:39244
I20260928 14:50:26.175642 17337 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb012d728f64714bf40de0c77e77331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:26 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59040
I20260928 14:50:26.175696 17475 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb012d728f64714bf40de0c77e77331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:26 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59142
I20260928 14:50:26.176237 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17132
I20260928 14:50:26.188853 17716 tablet_replica.cc:330] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: stopping tablet replica
I20260928 14:50:26.189118 17716 raft_consensus.cc:2229] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:26.189507 17716 raft_consensus.cc:2258] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:26.191877 17716 ts_tablet_manager.cc:1905] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:26.201344 17716 ts_tablet_manager.cc:1918] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:50:26.201640 17716 log.cc:1196] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/9cb012d728f64714bf40de0c77e77331
I20260928 14:50:26.202316 17716 ts_tablet_manager.cc:1939] T 9cb012d728f64714bf40de0c77e77331 P 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5: Deleting consensus metadata
I20260928 14:50:26.203662 17045 catalog_manager.cc:4841] TS 1a9e8e9e8a9b4d0db1fdcbc88dbfa5e5 (127.14.241.195:35457): tablet 9cb012d728f64714bf40de0c77e77331 (table test-table [id=07beed6142844436bd6f18e23c32fc4c]) successfully deleted
I20260928 14:50:26.203783 17714 tablet_replica.cc:330] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30: stopping tablet replica
I20260928 14:50:26.206602 17715 tablet_replica.cc:330] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c: stopping tablet replica
I20260928 14:50:26.206926 17715 raft_consensus.cc:2229] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:50:26.207324 17715 raft_consensus.cc:2258] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:26.208381 17715 ts_tablet_manager.cc:1905] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:26.208740 17714 raft_consensus.cc:2229] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:26.209118 17714 raft_consensus.cc:2258] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:26.210210 17714 ts_tablet_manager.cc:1905] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 14:50:26 dist-test-slave-vk40 krb5kdc[17016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607023, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:26.216629 17715 ts_tablet_manager.cc:1918] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:50:26.216910 17715 log.cc:1196] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/9cb012d728f64714bf40de0c77e77331
I20260928 14:50:26.217585 17715 ts_tablet_manager.cc:1939] T 9cb012d728f64714bf40de0c77e77331 P 376cfe628b4343829c7fe5d4f29e756c: Deleting consensus metadata
I20260928 14:50:26.218845 17714 ts_tablet_manager.cc:1918] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:50:26.218870 17048 catalog_manager.cc:4841] TS 376cfe628b4343829c7fe5d4f29e756c (127.14.241.194:33021): tablet 9cb012d728f64714bf40de0c77e77331 (table test-table [id=07beed6142844436bd6f18e23c32fc4c]) successfully deleted
I20260928 14:50:26.219048 17714 log.cc:1196] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/9cb012d728f64714bf40de0c77e77331
I20260928 14:50:26.219615 17714 ts_tablet_manager.cc:1939] T 9cb012d728f64714bf40de0c77e77331 P 882926d3969841bf8e0cb9fcb4172e30: Deleting consensus metadata
I20260928 14:50:26.220744 17048 catalog_manager.cc:4841] TS 882926d3969841bf8e0cb9fcb4172e30 (127.14.241.193:40707): tablet 9cb012d728f64714bf40de0c77e77331 (table test-table [id=07beed6142844436bd6f18e23c32fc4c]) successfully deleted
I20260928 14:50:26.224514 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:26.179542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60304 (local address 127.14.241.193:40707)
0928 14:50:26.202965 (+ 23423us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:26.202971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:26.202999 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:50:26.203070 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:26.203076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:26.203158 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:26.203280 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:26.205596 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:26.206476 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:26.207903 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:26.208812 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:26.208943 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:26.215612 (+ 6669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:26.215631 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:26.215647 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:26.215682 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:26.221047 (+ 5365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:26.221770 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:26.221779 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:26.221783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:26.221987 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:26.222465 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:26.222469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:26.222470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:26.222974 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:26.223372 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:26.223575 (+ 203us) server_negotiation.cc:299] Negotiation successful
0928 14:50:26.223846 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23260,"thread_start_us":25994,"threads_started":1}
W20260928 14:50:26.225383 17263 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60304
W20260928 14:50:26.226083 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17132: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:50:26.226447 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 17132
I20260928 14:50:26.249513 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17272
Sep 28 14:50:26 dist-test-slave-vk40 krb5kdc[17016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607023, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:50:26.266090 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:26.250257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39434 (local address 127.14.241.194:33021)
0928 14:50:26.251051 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:26.251055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:26.251072 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:50:26.251125 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:26.251129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:26.251232 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:26.251343 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:26.252692 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:26.253980 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:26.255190 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:26.255971 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:26.256087 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:26.260497 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:26.260514 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:26.260531 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:26.260571 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:26.263372 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:26.264059 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:26.264066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:26.264069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:26.264198 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:26.264578 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:26.264582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:26.264584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:26.264943 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:26.265226 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:26.265542 (+ 316us) server_negotiation.cc:299] Negotiation successful
0928 14:50:26.265777 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":452,"threads_started":1}
W20260928 14:50:26.266803 17402 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39434
W20260928 14:50:26.267558 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17272: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:50:26.267704 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 17272
I20260928 14:50:26.286140 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17411
Sep 28 14:50:26 dist-test-slave-vk40 krb5kdc[17016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607023, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:50:26.301031 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:26.286935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38768 (local address 127.14.241.195:35457)
0928 14:50:26.287498 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:26.287503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:26.287518 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:50:26.287572 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:26.287575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:26.287632 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:26.287739 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:26.289071 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:26.289822 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:26.291002 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:26.291408 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:26.291515 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:26.295595 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:26.295610 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:26.295625 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:26.295665 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:26.298337 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:26.298977 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:26.298984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:26.298986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:26.299150 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:26.299508 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:26.299512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:26.299514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:26.299886 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:26.300194 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:26.300469 (+ 275us) server_negotiation.cc:299] Negotiation successful
0928 14:50:26.300697 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":324,"threads_started":1}
W20260928 14:50:26.301674 17541 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38768
W20260928 14:50:26.302292 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17411: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:50:26.302464 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 17411
I20260928 14:50:26.320218 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17029
I20260928 14:50:26.321390 17101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:26.439919 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 17029
2026-09-28T14:50:26Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8529 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:50:26 dist-test-slave-vk40 krb5kdc[17724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:50:26 dist-test-slave-vk40 krb5kdc[17724](info): set up 2 sockets
Sep 28 14:50:26 dist-test-slave-vk40 krb5kdc[17724](info): commencing operation
krb5kdc: starting...
W20260928 14:50:27.443480 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.963s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:50:27 dist-test-slave-vk40 krb5kdc[17724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:50:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:50:27.628065 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46223
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:46223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:27.943742 17737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:27.944064 17737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:27.944192 17737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:27.951653 17737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:27.951776 17737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:27.951850 17737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:27.951931 17737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:27.951987 17737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:27.962036 17737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:46223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:27.963376 17737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:27.964744 17737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:27.976722 17742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:27.976650 17745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:27.976919 17737 server_base.cc:1034] running on GCE node
W20260928 14:50:27.976677 17743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:27.978021 17737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:27.979449 17737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:27.980705 17737 hybrid_clock.cc:648] HybridClock initialized: now 1790607027980645 us; error 34 us; skew 500 ppm
I20260928 14:50:27.981053 17737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:27 dist-test-slave-vk40 krb5kdc[17724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607027, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:27.984004 17737 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:50:27.985474 17737 webserver.cc:460] Webserver started at http://127.14.241.254:43287/ using document root <none> and password file <none>
I20260928 14:50:27.986129 17737 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:27.986225 17737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:27.986491 17737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:27.989550 17737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "7f6644a316bd408aad8eb9eda6d02963"
format_stamp: "Formatted at 2026-09-28 14:50:27 on dist-test-slave-vk40"
server_key: "b8d440e4ae15e8e0df3218918c5361ce"
server_key_iv: "eabf50ba4675b35968c87f5072a3ccc5"
server_key_version: "encryptionkey@0"
I20260928 14:50:27.990285 17737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "7f6644a316bd408aad8eb9eda6d02963"
format_stamp: "Formatted at 2026-09-28 14:50:27 on dist-test-slave-vk40"
server_key: "b8d440e4ae15e8e0df3218918c5361ce"
server_key_iv: "eabf50ba4675b35968c87f5072a3ccc5"
server_key_version: "encryptionkey@0"
I20260928 14:50:27.995465 17737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:50:27.998783 17752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:28.000036 17737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:50:28.000190 17737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "7f6644a316bd408aad8eb9eda6d02963"
format_stamp: "Formatted at 2026-09-28 14:50:27 on dist-test-slave-vk40"
server_key: "b8d440e4ae15e8e0df3218918c5361ce"
server_key_iv: "eabf50ba4675b35968c87f5072a3ccc5"
server_key_version: "encryptionkey@0"
I20260928 14:50:28.000408 17737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:28.014138 17737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:28.018589 17737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:28.018867 17737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:28.034526 17737 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:46223
I20260928 14:50:28.034546 17803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:46223 every 8 connection(s)
I20260928 14:50:28.035879 17737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:28.039636 17804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:28.044092 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 17737
I20260928 14:50:28.044312 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:28.044898 15303 external_mini_cluster.cc:1428] Setting key 92fe6ace843fc2caf51832bba6794be4
I20260928 14:50:28.050258 17804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963: Bootstrap starting.
I20260928 14:50:28.053787 17804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:28.055022 17804 log.cc:824] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:28.058291 17804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963: No bootstrap required, opened a new log
Sep 28 14:50:28 dist-test-slave-vk40 krb5kdc[17724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:28.066955 17804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6644a316bd408aad8eb9eda6d02963" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46223 } }
I20260928 14:50:28.067329 17804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:28.067386 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:28.046791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35200 (local address 127.14.241.254:46223)
0928 14:50:28.047630 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:28.047641 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:28.047684 (+ 43us) server_negotiation.cc:407] Connection header received
0928 14:50:28.049451 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:28.049475 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:28.049879 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:28.050204 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:28.051882 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:28.053203 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:28.054916 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:28.055639 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:28.055773 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:28.060173 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:28.060183 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:28.060198 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:28.060236 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:28.063317 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:28.064064 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:28.064070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:28.064077 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:28.064248 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:28.064680 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:28.064683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:28.064685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:28.065250 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:28.065495 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:28.065895 (+ 400us) server_negotiation.cc:299] Negotiation successful
0928 14:50:28.066272 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":345,"threads_started":1}
I20260928 14:50:28.067442 17804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f6644a316bd408aad8eb9eda6d02963, State: Initialized, Role: FOLLOWER
I20260928 14:50:28.068043 17804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6644a316bd408aad8eb9eda6d02963" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46223 } }
I20260928 14:50:28.068269 17804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:28.068424 17804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:28.068598 17804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:28.071527 17804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6644a316bd408aad8eb9eda6d02963" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46223 } }
I20260928 14:50:28.072031 17804 leader_election.cc:302] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f6644a316bd408aad8eb9eda6d02963; no voters:
I20260928 14:50:28.072598 17804 leader_election.cc:288] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:28.072710 17809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:28.073333 17809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [term 1 LEADER]: Becoming Leader. State: Replica: 7f6644a316bd408aad8eb9eda6d02963, State: Running, Role: LEADER
I20260928 14:50:28.073822 17809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6644a316bd408aad8eb9eda6d02963" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46223 } }
I20260928 14:50:28.074586 17804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:28.076617 17811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f6644a316bd408aad8eb9eda6d02963. Latest consensus state: current_term: 1 leader_uuid: "7f6644a316bd408aad8eb9eda6d02963" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6644a316bd408aad8eb9eda6d02963" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46223 } } }
I20260928 14:50:28.076536 17810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f6644a316bd408aad8eb9eda6d02963" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f6644a316bd408aad8eb9eda6d02963" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46223 } } }
I20260928 14:50:28.077142 17811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:28.077363 17810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:28.080579 17815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:28.085683 17815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:28.094722 17815 catalog_manager.cc:1347] Generated new cluster ID: 53691418a95e47c5a4d47f1526f4fb2e
I20260928 14:50:28.094832 17815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:28.125483 17815 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:28.126821 17815 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:28.138340 17815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f6644a316bd408aad8eb9eda6d02963: Generated new TSK 0
I20260928 14:50:28.139441 17815 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:50:28.192137 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46223
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:28.532533 17830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:28.532826 17830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:28.532920 17830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:28.540074 17830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:28.540185 17830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:28.540316 17830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:28.549862 17830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:28.551105 17830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:28.552382 17830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:28.564944 17835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:28.564899 17838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:28.564968 17836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:28.565908 17830 server_base.cc:1034] running on GCE node
I20260928 14:50:28.566515 17830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:28.567444 17830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:28.568634 17830 hybrid_clock.cc:648] HybridClock initialized: now 1790607028568580 us; error 47 us; skew 500 ppm
I20260928 14:50:28.568945 17830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:28 dist-test-slave-vk40 krb5kdc[17724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607028, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:28.572042 17830 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:50:28.573513 17830 webserver.cc:460] Webserver started at http://127.14.241.193:39333/ using document root <none> and password file <none>
I20260928 14:50:28.574186 17830 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:28.574299 17830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:28.574613 17830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:28.577636 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "d886e6ab0c8544debb3624dd7439840a"
format_stamp: "Formatted at 2026-09-28 14:50:28 on dist-test-slave-vk40"
server_key: "5a537e9463d2c10d7d5c33cc008ce567"
server_key_iv: "866000287790881540887c05c2f3a6ed"
server_key_version: "encryptionkey@0"
I20260928 14:50:28.578389 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "d886e6ab0c8544debb3624dd7439840a"
format_stamp: "Formatted at 2026-09-28 14:50:28 on dist-test-slave-vk40"
server_key: "5a537e9463d2c10d7d5c33cc008ce567"
server_key_iv: "866000287790881540887c05c2f3a6ed"
server_key_version: "encryptionkey@0"
I20260928 14:50:28.583770 17830 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:50:28.587248 17845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:28.588718 17830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:28.588877 17830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "d886e6ab0c8544debb3624dd7439840a"
format_stamp: "Formatted at 2026-09-28 14:50:28 on dist-test-slave-vk40"
server_key: "5a537e9463d2c10d7d5c33cc008ce567"
server_key_iv: "866000287790881540887c05c2f3a6ed"
server_key_version: "encryptionkey@0"
I20260928 14:50:28.589054 17830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:28.618009 17830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:28.622119 17830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:28.622390 17830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:28.623317 17830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:28.624645 17830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:28.624724 17830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:28.624840 17830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:28.624889 17830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:28.653906 17830 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:37719
I20260928 14:50:28.653985 17957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:37719 every 8 connection(s)
I20260928 14:50:28.655138 17830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:28.662880 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 17830
I20260928 14:50:28.663079 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:50:28.663895 15303 external_mini_cluster.cc:1428] Setting key 707954be49f8eb27577619e62aa6cf4d
Sep 28 14:50:28 dist-test-slave-vk40 krb5kdc[17724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607028, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:28.676146 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:28.657558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:34155 (local address 127.14.241.254:46223)
0928 14:50:28.657984 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:28.657989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:28.658912 (+ 923us) server_negotiation.cc:407] Connection header received
0928 14:50:28.660743 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:28.660748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:28.660818 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:28.660916 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:28.662363 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:28.663149 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:28.665243 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:28.665804 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:28.666029 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:28.670134 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:28.670139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:28.670143 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:28.670177 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:28.673046 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:28.673738 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:28.673743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:28.673746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:28.673913 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:28.674353 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:28.674358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:28.674361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:28.674702 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:28.674856 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:28.675553 (+ 697us) server_negotiation.cc:299] Negotiation successful
0928 14:50:28.675776 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":208,"threads_started":1}
I20260928 14:50:28.677031 17961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:28.658123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46223 (local address 127.14.241.193:34155)
0928 14:50:28.658762 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:28.658800 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:28.660495 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:28.661103 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:28.661111 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:28.661600 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:28.662197 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:28.662214 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:28.663320 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:28.663326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:28.665115 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:28.665126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:28.666153 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:28.666161 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:28.666470 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:28.667184 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:28.667232 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:28.669883 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:28.673184 (+ 3301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:28.673191 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:28.673211 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:28.673616 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:28.674038 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:28.674041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:28.674044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:28.674237 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:28.674803 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:28.674809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:28.675174 (+ 365us) client_negotiation.cc:769] Sending connection context
0928 14:50:28.675411 (+ 237us) client_negotiation.cc:240] Negotiation successful
0928 14:50:28.675831 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":240,"threads_started":1}
I20260928 14:50:28.678627 17958 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46223
I20260928 14:50:28.678980 17958 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:28.679883 17958 heartbeater.cc:511] Master 127.14.241.254:46223 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
I20260928 14:50:28.682283 17769 ts_manager.cc:194] Registered new tserver with Master: d886e6ab0c8544debb3624dd7439840a (127.14.241.193:37719)
I20260928 14:50:28.683779 17769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:34155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:50:28.705338 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46223
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:29.013343 17964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:29.013616 17964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:29.013715 17964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:29.020967 17964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:29.021077 17964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:29.021191 17964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:29.031136 17964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:29.032423 17964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:29.033699 17964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:29.048445 17969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:29.048419 17970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:29.049196 17972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:29.049517 17964 server_base.cc:1034] running on GCE node
I20260928 14:50:29.050143 17964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:29.051090 17964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:29.052309 17964 hybrid_clock.cc:648] HybridClock initialized: now 1790607029052228 us; error 54 us; skew 500 ppm
I20260928 14:50:29.052630 17964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:29 dist-test-slave-vk40 krb5kdc[17724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607029, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:29.058876 17964 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:50:29.060386 17964 webserver.cc:460] Webserver started at http://127.14.241.194:44119/ using document root <none> and password file <none>
I20260928 14:50:29.061070 17964 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:29.061182 17964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:29.061475 17964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:29.064388 17964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "08ac8339aa5f419990b6214528843f03"
format_stamp: "Formatted at 2026-09-28 14:50:29 on dist-test-slave-vk40"
server_key: "0ba318ba2db75d5dfea7a75ef74bc3c7"
server_key_iv: "78a34ed1e59a3897c08fc7112f37b563"
server_key_version: "encryptionkey@0"
I20260928 14:50:29.065155 17964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "08ac8339aa5f419990b6214528843f03"
format_stamp: "Formatted at 2026-09-28 14:50:29 on dist-test-slave-vk40"
server_key: "0ba318ba2db75d5dfea7a75ef74bc3c7"
server_key_iv: "78a34ed1e59a3897c08fc7112f37b563"
server_key_version: "encryptionkey@0"
I20260928 14:50:29.070439 17964 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:50:29.074208 17979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:29.075827 17964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:50:29.075987 17964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "08ac8339aa5f419990b6214528843f03"
format_stamp: "Formatted at 2026-09-28 14:50:29 on dist-test-slave-vk40"
server_key: "0ba318ba2db75d5dfea7a75ef74bc3c7"
server_key_iv: "78a34ed1e59a3897c08fc7112f37b563"
server_key_version: "encryptionkey@0"
I20260928 14:50:29.076217 17964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:29.094763 17964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:29.099097 17964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:29.099382 17964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:29.100363 17964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:29.101774 17964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:29.101852 17964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:29.101970 17964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:29.102020 17964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:29.136274 17964 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:35781
I20260928 14:50:29.136341 18091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:35781 every 8 connection(s)
I20260928 14:50:29.137565 17964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:29.141659 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 17964
I20260928 14:50:29.141855 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:50:29.142382 15303 external_mini_cluster.cc:1428] Setting key 21893290079d7777d48d8d74dd61e9ed
Sep 28 14:50:29 dist-test-slave-vk40 krb5kdc[17724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607029, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:29.158496 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:29.140062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:58475 (local address 127.14.241.254:46223)
0928 14:50:29.140223 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:29.140228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:29.141571 (+ 1343us) server_negotiation.cc:407] Connection header received
0928 14:50:29.143801 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:29.143808 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:29.143886 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:29.143995 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:29.146014 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:29.146731 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:29.148910 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:29.149338 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:29.149429 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:29.152949 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:29.152954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:29.152957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:29.152993 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:29.155457 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:29.156170 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:29.156174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:29.156177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:29.156345 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:29.156769 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:29.156773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:29.156775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:29.157059 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:29.157191 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:29.157923 (+ 732us) server_negotiation.cc:299] Negotiation successful
0928 14:50:29.158163 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
I20260928 14:50:29.159521 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:29.140607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46223 (local address 127.14.241.194:58475)
0928 14:50:29.141344 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:29.141417 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:29.143495 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:29.144205 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:29.144215 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:29.144973 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:29.145813 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:29.145841 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:29.146887 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:29.146892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:29.148749 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:29.148765 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:29.149553 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:29.149560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:29.149838 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:29.150534 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:29.150562 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:29.152697 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:29.155600 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:29.155609 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:29.155626 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:29.156041 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:29.156453 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:29.156458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:29.156462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:29.156653 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:29.157183 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:29.157191 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:29.157552 (+ 361us) client_negotiation.cc:769] Sending connection context
0928 14:50:29.157794 (+ 242us) client_negotiation.cc:240] Negotiation successful
0928 14:50:29.158342 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":258,"threads_started":1}
I20260928 14:50:29.160969 18092 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46223
I20260928 14:50:29.161275 18092 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
I20260928 14:50:29.162032 18092 heartbeater.cc:511] Master 127.14.241.254:46223 requested a full tablet report, sending...
I20260928 14:50:29.163458 17769 ts_manager.cc:194] Registered new tserver with Master: 08ac8339aa5f419990b6214528843f03 (127.14.241.194:35781)
I20260928 14:50:29.164255 17769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:58475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:50:29.183332 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46223
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:29.494938 18097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:29.495226 18097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:29.495348 18097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:29.502719 18097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:29.502841 18097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:29.502985 18097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:29.513118 18097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:29.514464 18097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:29.515830 18097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:29.529739 18105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:29.529672 18102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:29.529755 18103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:29.533869 18097 server_base.cc:1034] running on GCE node
I20260928 14:50:29.534523 18097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:29.535521 18097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:29.536707 18097 hybrid_clock.cc:648] HybridClock initialized: now 1790607029536657 us; error 42 us; skew 500 ppm
I20260928 14:50:29.537019 18097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:29 dist-test-slave-vk40 krb5kdc[17724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607029, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:29.540059 18097 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:50:29.541581 18097 webserver.cc:460] Webserver started at http://127.14.241.195:39151/ using document root <none> and password file <none>
I20260928 14:50:29.542268 18097 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:29.542384 18097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:29.542670 18097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:29.545549 18097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "8804380c7a8843e2b1bdc0c4d3143cad"
format_stamp: "Formatted at 2026-09-28 14:50:29 on dist-test-slave-vk40"
server_key: "a2d701620e4e01810532b330aee4da88"
server_key_iv: "57f810c91c797d27b9afe2062ea1dd1d"
server_key_version: "encryptionkey@0"
I20260928 14:50:29.546303 18097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "8804380c7a8843e2b1bdc0c4d3143cad"
format_stamp: "Formatted at 2026-09-28 14:50:29 on dist-test-slave-vk40"
server_key: "a2d701620e4e01810532b330aee4da88"
server_key_iv: "57f810c91c797d27b9afe2062ea1dd1d"
server_key_version: "encryptionkey@0"
I20260928 14:50:29.551352 18097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:50:29.554637 18112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:29.555910 18097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:50:29.556051 18097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "8804380c7a8843e2b1bdc0c4d3143cad"
format_stamp: "Formatted at 2026-09-28 14:50:29 on dist-test-slave-vk40"
server_key: "a2d701620e4e01810532b330aee4da88"
server_key_iv: "57f810c91c797d27b9afe2062ea1dd1d"
server_key_version: "encryptionkey@0"
I20260928 14:50:29.556226 18097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:29.575531 18097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:29.579380 18097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:29.579654 18097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:29.580763 18097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:29.582266 18097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:29.582345 18097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:29.582430 18097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:29.582472 18097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:29.611984 18097 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:35477
I20260928 14:50:29.612031 18224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:35477 every 8 connection(s)
I20260928 14:50:29.613241 18097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:29.620683 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 18097
I20260928 14:50:29.620889 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:50:29.621276 15303 external_mini_cluster.cc:1428] Setting key 88fd2b4824642bab2f18991a84cef0a2
Sep 28 14:50:29 dist-test-slave-vk40 krb5kdc[17724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607029, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:29.632812 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:29.615723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:33901 (local address 127.14.241.254:46223)
0928 14:50:29.615907 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:29.615912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:29.617105 (+ 1193us) server_negotiation.cc:407] Connection header received
0928 14:50:29.618902 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:29.618908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:29.618991 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:29.619077 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:29.620793 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:29.621604 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:29.623481 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:29.623953 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:29.624066 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:29.627502 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:29.627506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:29.627510 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:29.627541 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:29.630050 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:29.630757 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:29.630761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:29.630763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:29.630896 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:29.631284 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:29.631288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:29.631290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:29.631596 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:29.631713 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:29.632402 (+ 689us) server_negotiation.cc:299] Negotiation successful
0928 14:50:29.632573 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 14:50:29.633761 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:29.616159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46223 (local address 127.14.241.195:33901)
0928 14:50:29.616926 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:29.616971 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:29.618616 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:29.619326 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:29.619335 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:29.619860 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:29.620560 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:29.620588 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:29.621761 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:29.621765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:29.623341 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:29.623358 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:29.624399 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:29.624406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:29.624609 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:29.625243 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:29.625271 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:29.627301 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:29.630187 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:29.630194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:29.630208 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:29.630645 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:29.630977 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:29.630979 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:29.630982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:29.631155 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:29.631711 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:29.631717 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:29.632044 (+ 327us) client_negotiation.cc:769] Sending connection context
0928 14:50:29.632266 (+ 222us) client_negotiation.cc:240] Negotiation successful
0928 14:50:29.632706 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":280,"threads_started":1}
I20260928 14:50:29.635190 18225 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46223
I20260928 14:50:29.635501 18225 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:29.636305 18225 heartbeater.cc:511] Master 127.14.241.254:46223 requested a full tablet report, sending...
I20260928 14:50:29.637676 17769 ts_manager.cc:194] Registered new tserver with Master: 8804380c7a8843e2b1bdc0c4d3143cad (127.14.241.195:35477)
I20260928 14:50:29.638482 17769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:33901
I20260928 14:50:29.648821 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 14:50:29.676997 17960 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:50:29.674501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35210 (local address 127.14.241.254:46223)
0928 14:50:29.674682 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:29.674688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:29.675134 (+ 446us) server_negotiation.cc:407] Connection header received
0928 14:50:29.675278 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:29.675283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:29.675353 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:29.675466 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:29.676757 (+ 1291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35210: BlockingRecv error: recv got EOF from 127.0.0.1:35210 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:50:29.680011 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17830
W20260928 14:50:29.682726 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17830: Not authorized: Client connection negotiation failed: client connection to 127.14.241.193:37719: server requires authentication, but client does not have Kerberos credentials available
I20260928 14:50:29.682906 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 17830
I20260928 14:50:29.696218 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17964
W20260928 14:50:29.698756 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17964: Not authorized: Client connection negotiation failed: client connection to 127.14.241.194:35781: server requires authentication, but client does not have Kerberos credentials available
I20260928 14:50:29.698925 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 17964
I20260928 14:50:29.711606 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18097
W20260928 14:50:29.714304 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18097: Not authorized: Client connection negotiation failed: client connection to 127.14.241.195:35477: server requires authentication, but client does not have Kerberos credentials available
I20260928 14:50:29.714421 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 18097
I20260928 14:50:29.726426 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 17737
I20260928 14:50:29.727628 17799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:29.830693 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 17737
2026-09-28T14:50:29Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3387 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:50:29 dist-test-slave-vk40 krb5kdc[18242](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:50:29 dist-test-slave-vk40 krb5kdc[18242](info): set up 2 sockets
Sep 28 14:50:29 dist-test-slave-vk40 krb5kdc[18242](info): commencing operation
krb5kdc: starting...
W20260928 14:50:30.835153 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.002s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:50:30 dist-test-slave-vk40 krb5kdc[18242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:50:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:50:31.019524 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:42047
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:42047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:31.342864 18255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:31.343169 18255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:31.343298 18255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:31.350538 18255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:31.350666 18255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:31.350746 18255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:31.350826 18255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:31.350890 18255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:31.360833 18255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:42047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:42047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:31.362150 18255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:31.363481 18255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:31.374912 18263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:31.375000 18260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:31.374938 18261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:31.375725 18255 server_base.cc:1034] running on GCE node
I20260928 14:50:31.376542 18255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:31.377964 18255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:31.379149 18255 hybrid_clock.cc:648] HybridClock initialized: now 1790607031379108 us; error 37 us; skew 500 ppm
I20260928 14:50:31.379499 18255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:31 dist-test-slave-vk40 krb5kdc[18242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607031, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:31.382491 18255 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:50:31.383899 18255 webserver.cc:460] Webserver started at http://127.14.241.254:44027/ using document root <none> and password file <none>
I20260928 14:50:31.384624 18255 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:31.384735 18255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:31.385026 18255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:31.388023 18255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "29df9fd7bbde4a818aad60b62bc8f08d"
format_stamp: "Formatted at 2026-09-28 14:50:31 on dist-test-slave-vk40"
server_key: "5405d8b186e38082dd63b48e4aad474b"
server_key_iv: "df737497e080379ccc5a2adc8cd18225"
server_key_version: "encryptionkey@0"
I20260928 14:50:31.388835 18255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "29df9fd7bbde4a818aad60b62bc8f08d"
format_stamp: "Formatted at 2026-09-28 14:50:31 on dist-test-slave-vk40"
server_key: "5405d8b186e38082dd63b48e4aad474b"
server_key_iv: "df737497e080379ccc5a2adc8cd18225"
server_key_version: "encryptionkey@0"
I20260928 14:50:31.394057 18255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:50:31.397442 18270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:31.398741 18255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:50:31.398900 18255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "29df9fd7bbde4a818aad60b62bc8f08d"
format_stamp: "Formatted at 2026-09-28 14:50:31 on dist-test-slave-vk40"
server_key: "5405d8b186e38082dd63b48e4aad474b"
server_key_iv: "df737497e080379ccc5a2adc8cd18225"
server_key_version: "encryptionkey@0"
I20260928 14:50:31.399085 18255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:31.421815 18255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:31.426147 18255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:31.426422 18255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:31.443655 18255 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:42047
I20260928 14:50:31.443684 18321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:42047 every 8 connection(s)
I20260928 14:50:31.445056 18255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:31.447080 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 18255
I20260928 14:50:31.447283 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:31.447870 15303 external_mini_cluster.cc:1428] Setting key 7e2ff29bacc9aaa8f7499ea460876d61
I20260928 14:50:31.448982 18322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 14:50:31 dist-test-slave-vk40 krb5kdc[18242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:31.463081 18322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d: Bootstrap starting.
I20260928 14:50:31.466634 18322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:31.467923 18322 log.cc:824] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:31.470686 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:31.449640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49016 (local address 127.14.241.254:42047)
0928 14:50:31.450378 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:31.450389 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:31.450433 (+ 44us) server_negotiation.cc:407] Connection header received
0928 14:50:31.451999 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:31.452020 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:31.452466 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:31.452804 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:31.454666 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:31.456038 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:31.457710 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:31.458394 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:31.458537 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:31.463169 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:31.463177 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:31.463193 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:31.463249 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:31.466298 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:31.467150 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:31.467161 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:31.467167 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:31.467357 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:31.467819 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:31.467824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:31.467826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:31.468545 (+ 719us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:31.468903 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:31.469222 (+ 319us) server_negotiation.cc:299] Negotiation successful
0928 14:50:31.469659 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":345,"threads_started":1}
I20260928 14:50:31.470934 18322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d: No bootstrap required, opened a new log
I20260928 14:50:31.474743 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:50:31.480626 18322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29df9fd7bbde4a818aad60b62bc8f08d" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42047 } }
I20260928 14:50:31.481097 18322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:31.481191 18322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29df9fd7bbde4a818aad60b62bc8f08d, State: Initialized, Role: FOLLOWER
I20260928 14:50:31.481935 18322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29df9fd7bbde4a818aad60b62bc8f08d" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42047 } }
I20260928 14:50:31.482256 18322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:31.482416 18322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:31.482600 18322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:31.484786 18322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29df9fd7bbde4a818aad60b62bc8f08d" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42047 } }
I20260928 14:50:31.485285 18322 leader_election.cc:302] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29df9fd7bbde4a818aad60b62bc8f08d; no voters:
I20260928 14:50:31.485895 18322 leader_election.cc:288] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:31.486013 18328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:31.487072 18328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [term 1 LEADER]: Becoming Leader. State: Replica: 29df9fd7bbde4a818aad60b62bc8f08d, State: Running, Role: LEADER
I20260928 14:50:31.487668 18328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29df9fd7bbde4a818aad60b62bc8f08d" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42047 } }
I20260928 14:50:31.488214 18322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:31.491639 18330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29df9fd7bbde4a818aad60b62bc8f08d. Latest consensus state: current_term: 1 leader_uuid: "29df9fd7bbde4a818aad60b62bc8f08d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29df9fd7bbde4a818aad60b62bc8f08d" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42047 } } }
I20260928 14:50:31.492053 18330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:31.497563 18329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29df9fd7bbde4a818aad60b62bc8f08d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29df9fd7bbde4a818aad60b62bc8f08d" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42047 } } }
I20260928 14:50:31.497819 18329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:31.498313 18335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:31.503261 18335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:31.526139 18335 catalog_manager.cc:1347] Generated new cluster ID: c768837d2c2144d8aecdb7c218e27491
I20260928 14:50:31.526268 18335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:31.552980 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:31.545435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.14.241.254:42047)
0928 14:50:31.545628 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:31.545635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:31.546194 (+ 559us) server_negotiation.cc:407] Connection header received
0928 14:50:31.546381 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:31.546386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:31.546456 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:31.546543 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:31.547558 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:31.548417 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:31.549696 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:31.550118 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:31.550226 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:31.551326 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:31.551345 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:31.551350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:31.552084 (+ 734us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:31.552146 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:31.552181 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:31.552321 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:31.552483 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 14:50:31.552588 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":66}
I20260928 14:50:31.553114 18335 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:31.555049 18335 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:31.564826 18335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29df9fd7bbde4a818aad60b62bc8f08d: Generated new TSK 0
I20260928 14:50:31.565682 18335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:50:31.574580 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18255
I20260928 14:50:31.575991 18317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:31.666795 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 18255
2026-09-28T14:50:31Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1831 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:50:31 dist-test-slave-vk40 krb5kdc[18356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:50:31 dist-test-slave-vk40 krb5kdc[18356](info): set up 2 sockets
Sep 28 14:50:31 dist-test-slave-vk40 krb5kdc[18356](info): commencing operation
krb5kdc: starting...
W20260928 14:50:32.655112 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.008s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:50:32 dist-test-slave-vk40 krb5kdc[18356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:50:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:50:32.836834 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:43573
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:43573
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:33.156002 18369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:33.156313 18369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:33.156435 18369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:33.163635 18369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:33.163739 18369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:33.163784 18369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:33.163844 18369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:33.163893 18369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:33.173704 18369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:43573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:43573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:33.175006 18369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:33.176265 18369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:33.187502 18377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:33.187574 18374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:33.187541 18375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:33.188416 18369 server_base.cc:1034] running on GCE node
I20260928 14:50:33.189196 18369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:33.190661 18369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:33.191892 18369 hybrid_clock.cc:648] HybridClock initialized: now 1790607033191825 us; error 50 us; skew 500 ppm
I20260928 14:50:33.192243 18369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:33 dist-test-slave-vk40 krb5kdc[18356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607033, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:33.195363 18369 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:50:33.197049 18369 webserver.cc:460] Webserver started at http://127.14.241.254:33587/ using document root <none> and password file <none>
I20260928 14:50:33.197760 18369 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:33.197892 18369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:33.198191 18369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:33.201375 18369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "a5ca01e4cca747c082eee8b1f6af2b4f"
format_stamp: "Formatted at 2026-09-28 14:50:33 on dist-test-slave-vk40"
server_key: "2e27ec391a50f5d7546314bd5749b018"
server_key_iv: "104402f65f3625606d8575fe49f65779"
server_key_version: "encryptionkey@0"
I20260928 14:50:33.202172 18369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "a5ca01e4cca747c082eee8b1f6af2b4f"
format_stamp: "Formatted at 2026-09-28 14:50:33 on dist-test-slave-vk40"
server_key: "2e27ec391a50f5d7546314bd5749b018"
server_key_iv: "104402f65f3625606d8575fe49f65779"
server_key_version: "encryptionkey@0"
I20260928 14:50:33.207499 18369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:50:33.211153 18384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:33.212695 18369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:33.212859 18369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "a5ca01e4cca747c082eee8b1f6af2b4f"
format_stamp: "Formatted at 2026-09-28 14:50:33 on dist-test-slave-vk40"
server_key: "2e27ec391a50f5d7546314bd5749b018"
server_key_iv: "104402f65f3625606d8575fe49f65779"
server_key_version: "encryptionkey@0"
I20260928 14:50:33.213063 18369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:33.236117 18369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:33.240384 18369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:33.240650 18369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:33.258388 18369 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:43573
I20260928 14:50:33.258445 18435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:43573 every 8 connection(s)
I20260928 14:50:33.259750 18369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:33.263356 18436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:33.263397 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 18369
I20260928 14:50:33.263567 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:33.264171 15303 external_mini_cluster.cc:1428] Setting key 040dc613307adffd7e493e977d639a32
I20260928 14:50:33.276891 18436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f: Bootstrap starting.
Sep 28 14:50:33 dist-test-slave-vk40 krb5kdc[18356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:33.280846 18436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:33.282083 18436 log.cc:824] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:33.285099 18436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f: No bootstrap required, opened a new log
I20260928 14:50:33.287871 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:33.266217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49478 (local address 127.14.241.254:43573)
0928 14:50:33.266982 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:33.266994 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:33.267034 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:50:33.269059 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:33.269082 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:33.269522 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:33.269875 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:33.271492 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:33.272851 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:33.274653 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:33.275372 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:33.275512 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:33.280572 (+ 5060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:33.280581 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:33.280599 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:33.280645 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:33.283688 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:33.284405 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:33.284412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:33.284418 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:33.284599 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:33.284976 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:33.284980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:33.284982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:33.285569 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:33.285896 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:33.286329 (+ 433us) server_negotiation.cc:299] Negotiation successful
0928 14:50:33.286798 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":326,"threads_started":1}
I20260928 14:50:33.293198 18436 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca01e4cca747c082eee8b1f6af2b4f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43573 } }
I20260928 14:50:33.293560 18436 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:33.293656 18436 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5ca01e4cca747c082eee8b1f6af2b4f, State: Initialized, Role: FOLLOWER
I20260928 14:50:33.294180 18436 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca01e4cca747c082eee8b1f6af2b4f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43573 } }
I20260928 14:50:33.294385 18436 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:33.294514 18436 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:33.294653 18436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:33.297108 18436 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca01e4cca747c082eee8b1f6af2b4f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43573 } }
I20260928 14:50:33.297596 18436 leader_election.cc:302] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5ca01e4cca747c082eee8b1f6af2b4f; no voters:
I20260928 14:50:33.298122 18436 leader_election.cc:288] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:33.298262 18441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:33.298856 18441 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [term 1 LEADER]: Becoming Leader. State: Replica: a5ca01e4cca747c082eee8b1f6af2b4f, State: Running, Role: LEADER
I20260928 14:50:33.299396 18441 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca01e4cca747c082eee8b1f6af2b4f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43573 } }
I20260928 14:50:33.300110 18436 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:33.302091 18443 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5ca01e4cca747c082eee8b1f6af2b4f. Latest consensus state: current_term: 1 leader_uuid: "a5ca01e4cca747c082eee8b1f6af2b4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca01e4cca747c082eee8b1f6af2b4f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43573 } } }
I20260928 14:50:33.302152 18442 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5ca01e4cca747c082eee8b1f6af2b4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ca01e4cca747c082eee8b1f6af2b4f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43573 } } }
I20260928 14:50:33.302496 18442 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:33.302496 18443 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:33.305851 18448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:33.310149 18448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:33.318817 18448 catalog_manager.cc:1347] Generated new cluster ID: fc883cfb083f44fc942d5ca9395fd4c5
I20260928 14:50:33.318953 18448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:33.334991 18448 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:33.336766 18448 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:33.347539 18448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5ca01e4cca747c082eee8b1f6af2b4f: Generated new TSK 0
I20260928 14:50:33.348726 18448 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:50:33.396240 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:43573
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:33.717406 18462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:33.717733 18462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:33.717861 18462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:33.725181 18462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:33.725306 18462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:33.725467 18462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:33.735178 18462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:43573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:33.736532 18462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:33.737841 18462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:33.750471 18467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:33.750440 18468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:33.750525 18470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:33.751278 18462 server_base.cc:1034] running on GCE node
I20260928 14:50:33.751857 18462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:33.752833 18462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:33.754030 18462 hybrid_clock.cc:648] HybridClock initialized: now 1790607033753965 us; error 52 us; skew 500 ppm
I20260928 14:50:33.754343 18462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:33 dist-test-slave-vk40 krb5kdc[18356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607033, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:33.757385 18462 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:50:33.758766 18462 webserver.cc:460] Webserver started at http://127.14.241.193:45287/ using document root <none> and password file <none>
I20260928 14:50:33.759423 18462 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:33.759523 18462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:33.759768 18462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:33.762810 18462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "781e7cabad53469fb2b4e9b26c4425f0"
format_stamp: "Formatted at 2026-09-28 14:50:33 on dist-test-slave-vk40"
server_key: "0a53c3abbbfb3c49a3c3b62f8d44e97f"
server_key_iv: "690ef534612cc5dd3564077a91d3b976"
server_key_version: "encryptionkey@0"
I20260928 14:50:33.763585 18462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "781e7cabad53469fb2b4e9b26c4425f0"
format_stamp: "Formatted at 2026-09-28 14:50:33 on dist-test-slave-vk40"
server_key: "0a53c3abbbfb3c49a3c3b62f8d44e97f"
server_key_iv: "690ef534612cc5dd3564077a91d3b976"
server_key_version: "encryptionkey@0"
I20260928 14:50:33.771378 18462 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260928 14:50:33.775498 18477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:33.777006 18462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:33.777179 18462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "781e7cabad53469fb2b4e9b26c4425f0"
format_stamp: "Formatted at 2026-09-28 14:50:33 on dist-test-slave-vk40"
server_key: "0a53c3abbbfb3c49a3c3b62f8d44e97f"
server_key_iv: "690ef534612cc5dd3564077a91d3b976"
server_key_version: "encryptionkey@0"
I20260928 14:50:33.777388 18462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:33.807241 18462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:33.811354 18462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:33.811633 18462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:33.812582 18462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:33.813858 18462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:33.813932 18462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:33.814052 18462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:33.814112 18462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:33.844134 18462 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:35101
I20260928 14:50:33.844177 18589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:35101 every 8 connection(s)
I20260928 14:50:33.845438 18462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:33.855217 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 18462
I20260928 14:50:33.855400 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:50:33.856003 15303 external_mini_cluster.cc:1428] Setting key 2079e98191d1166389e99c05a76ec355
Sep 28 14:50:33 dist-test-slave-vk40 krb5kdc[18356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607033, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:33.864686 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:33.847930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:43435 (local address 127.14.241.254:43573)
0928 14:50:33.848358 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:33.848365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:33.849325 (+ 960us) server_negotiation.cc:407] Connection header received
0928 14:50:33.851111 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:33.851117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:33.851211 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:33.851304 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:33.852910 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:33.853628 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:33.855166 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:33.855622 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:33.855755 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:33.859155 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:33.859160 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:33.859164 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:33.859199 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:33.861609 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:33.862368 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:33.862373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:33.862375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:33.862497 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:33.862922 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:33.862926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:33.862928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:33.863204 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:33.863338 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:33.864070 (+ 732us) server_negotiation.cc:299] Negotiation successful
0928 14:50:33.864332 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":195,"threads_started":1}
I20260928 14:50:33.865728 18593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:33.848476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43573 (local address 127.14.241.193:43435)
0928 14:50:33.849157 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:33.849207 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:33.850841 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:33.851564 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:33.851573 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:33.852094 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:33.852742 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:33.852761 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:33.853779 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:33.853783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:33.855025 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:33.855038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:33.855847 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:33.855854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:33.856080 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:33.856768 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:33.856791 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:33.858933 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:33.861773 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:33.861783 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:33.861802 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:33.862244 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:33.862602 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:33.862607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:33.862611 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:33.862811 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:33.863328 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:33.863336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:33.863705 (+ 369us) client_negotiation.cc:769] Sending connection context
0928 14:50:33.863942 (+ 237us) client_negotiation.cc:240] Negotiation successful
0928 14:50:33.864575 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":265,"threads_started":1}
I20260928 14:50:33.867345 18590 heartbeater.cc:347] Connected to a master server at 127.14.241.254:43573
I20260928 14:50:33.867673 18590 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:33.868575 18590 heartbeater.cc:511] Master 127.14.241.254:43573 requested a full tablet report, sending...
I20260928 14:50:33.871145 18401 ts_manager.cc:194] Registered new tserver with Master: 781e7cabad53469fb2b4e9b26c4425f0 (127.14.241.193:35101)
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
I20260928 14:50:33.873162 18401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:43435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:50:33.899029 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:43573
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:34.213804 18596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:34.214084 18596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:34.214185 18596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:34.221544 18596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:34.221657 18596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:34.221767 18596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:34.231441 18596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:43573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:34.232712 18596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:34.233979 18596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:34.246469 18601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:34.246513 18604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:34.246537 18602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:34.247515 18596 server_base.cc:1034] running on GCE node
I20260928 14:50:34.248071 18596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:34.249006 18596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:34.250193 18596 hybrid_clock.cc:648] HybridClock initialized: now 1790607034250144 us; error 43 us; skew 500 ppm
I20260928 14:50:34.250494 18596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:34 dist-test-slave-vk40 krb5kdc[18356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607034, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:34.258911 18596 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:50:34.260692 18596 webserver.cc:460] Webserver started at http://127.14.241.194:43029/ using document root <none> and password file <none>
I20260928 14:50:34.261520 18596 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:34.261652 18596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:34.261956 18596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:34.265725 18596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "da324e4219944583a0026677c38b2fda"
format_stamp: "Formatted at 2026-09-28 14:50:34 on dist-test-slave-vk40"
server_key: "e577ade0321648004c76ed9f303600dc"
server_key_iv: "9b08c635f39f530e3f568d56518de1d6"
server_key_version: "encryptionkey@0"
I20260928 14:50:34.266650 18596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "da324e4219944583a0026677c38b2fda"
format_stamp: "Formatted at 2026-09-28 14:50:34 on dist-test-slave-vk40"
server_key: "e577ade0321648004c76ed9f303600dc"
server_key_iv: "9b08c635f39f530e3f568d56518de1d6"
server_key_version: "encryptionkey@0"
I20260928 14:50:34.272908 18596 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260928 14:50:34.277170 18611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:34.278555 18596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260928 14:50:34.278692 18596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "da324e4219944583a0026677c38b2fda"
format_stamp: "Formatted at 2026-09-28 14:50:34 on dist-test-slave-vk40"
server_key: "e577ade0321648004c76ed9f303600dc"
server_key_iv: "9b08c635f39f530e3f568d56518de1d6"
server_key_version: "encryptionkey@0"
I20260928 14:50:34.278873 18596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:34.298508 18596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:34.302598 18596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:34.302862 18596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:34.303790 18596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:34.305150 18596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:34.305222 18596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:34.305334 18596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:34.305385 18596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:34.335517 18596 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:38243
I20260928 14:50:34.335557 18723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:38243 every 8 connection(s)
I20260928 14:50:34.336779 18596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:34.346632 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 18596
I20260928 14:50:34.346814 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:50:34.347309 15303 external_mini_cluster.cc:1428] Setting key cf5d87ca183c622a665cc7b51a1c2af6
Sep 28 14:50:34 dist-test-slave-vk40 krb5kdc[18356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607034, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:34.356652 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:34.339236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:44315 (local address 127.14.241.254:43573)
0928 14:50:34.339421 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:34.339426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:34.340493 (+ 1067us) server_negotiation.cc:407] Connection header received
0928 14:50:34.342256 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:34.342261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:34.342348 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:34.342439 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:34.343980 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.344784 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.346456 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.346930 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.347049 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:34.350883 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:34.350890 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:34.350893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:34.350928 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:34.353455 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.354258 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.354264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.354267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.354406 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.354845 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.354848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.354850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.355186 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:34.355324 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:34.356033 (+ 709us) server_negotiation.cc:299] Negotiation successful
0928 14:50:34.356306 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 14:50:34.357674 18726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:34.339622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43573 (local address 127.14.241.194:44315)
0928 14:50:34.340335 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:34.340379 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:34.341955 (+ 1576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:34.342656 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:34.342665 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:34.343186 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:34.343773 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:34.343791 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.344901 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.344905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:34.346299 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:34.346321 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.347150 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.347158 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:34.347391 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:34.348166 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:34.348194 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:34.350620 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:34.353611 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:34.353621 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:34.353640 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:34.354136 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:34.354542 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:34.354545 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:34.354547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:34.354736 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:34.355314 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:34.355322 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:34.355682 (+ 360us) client_negotiation.cc:769] Sending connection context
0928 14:50:34.355938 (+ 256us) client_negotiation.cc:240] Negotiation successful
0928 14:50:34.356542 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":255,"threads_started":1}
I20260928 14:50:34.359268 18724 heartbeater.cc:347] Connected to a master server at 127.14.241.254:43573
I20260928 14:50:34.359575 18724 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:34.360351 18724 heartbeater.cc:511] Master 127.14.241.254:43573 requested a full tablet report, sending...
I20260928 14:50:34.361900 18401 ts_manager.cc:194] Registered new tserver with Master: da324e4219944583a0026677c38b2fda (127.14.241.194:38243)
I20260928 14:50:34.362722 18401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:44315
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:50:34.387840 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:43573
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:34.701423 18729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:34.701736 18729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:34.701836 18729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:34.709002 18729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:34.709122 18729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:34.709236 18729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:34.718981 18729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:43573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:34.720198 18729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:34.721567 18729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:34.734017 18735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:34.733981 18734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:34.734064 18737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:34.734872 18729 server_base.cc:1034] running on GCE node
I20260928 14:50:34.735488 18729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:34.736476 18729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:34.737670 18729 hybrid_clock.cc:648] HybridClock initialized: now 1790607034737622 us; error 45 us; skew 500 ppm
I20260928 14:50:34.737962 18729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:34 dist-test-slave-vk40 krb5kdc[18356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607034, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:34.740922 18729 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:50:34.742239 18729 webserver.cc:460] Webserver started at http://127.14.241.195:35321/ using document root <none> and password file <none>
I20260928 14:50:34.742861 18729 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:34.742956 18729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:34.743191 18729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:34.746006 18729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "45e65d0c35034617920a02af42872591"
format_stamp: "Formatted at 2026-09-28 14:50:34 on dist-test-slave-vk40"
server_key: "c5a80ff866a60396c77372583829eec5"
server_key_iv: "3f5e42ce17b6a798cef23383e64afc5f"
server_key_version: "encryptionkey@0"
I20260928 14:50:34.746730 18729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "45e65d0c35034617920a02af42872591"
format_stamp: "Formatted at 2026-09-28 14:50:34 on dist-test-slave-vk40"
server_key: "c5a80ff866a60396c77372583829eec5"
server_key_iv: "3f5e42ce17b6a798cef23383e64afc5f"
server_key_version: "encryptionkey@0"
I20260928 14:50:34.751637 18729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260928 14:50:34.755138 18744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:34.756474 18729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260928 14:50:34.756618 18729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "45e65d0c35034617920a02af42872591"
format_stamp: "Formatted at 2026-09-28 14:50:34 on dist-test-slave-vk40"
server_key: "c5a80ff866a60396c77372583829eec5"
server_key_iv: "3f5e42ce17b6a798cef23383e64afc5f"
server_key_version: "encryptionkey@0"
I20260928 14:50:34.756788 18729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:34.771376 18729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:34.775296 18729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:34.775557 18729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:34.776593 18729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:34.778107 18729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:34.778184 18729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:34.778316 18729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:34.778375 18729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:34.807859 18729 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:36061
I20260928 14:50:34.807893 18856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:36061 every 8 connection(s)
I20260928 14:50:34.809120 18729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:34.815544 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 18729
I20260928 14:50:34.815732 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:50:34.816313 15303 external_mini_cluster.cc:1428] Setting key ef8225d24c8c29bced5958721203c4ef
Sep 28 14:50:34 dist-test-slave-vk40 krb5kdc[18356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607034, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:34.828933 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:34.811606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:51777 (local address 127.14.241.254:43573)
0928 14:50:34.811852 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:34.811859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:34.813001 (+ 1142us) server_negotiation.cc:407] Connection header received
0928 14:50:34.814748 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:34.814754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:34.814826 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:34.814918 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:34.817001 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.818046 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.819876 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.820308 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.820420 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:34.823770 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:34.823775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:34.823778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:34.823814 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:34.826150 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.826860 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.826864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.826867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.826997 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.827379 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.827382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.827384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.827661 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:34.827816 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:34.828497 (+ 681us) server_negotiation.cc:299] Negotiation successful
0928 14:50:34.828670 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 14:50:34.829998 18859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:34.812092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43573 (local address 127.14.241.195:51777)
0928 14:50:34.812837 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:34.812881 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:34.814467 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:34.815149 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:34.815160 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:34.815931 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:34.816797 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:34.816825 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.818223 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.818228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:34.819751 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:34.819765 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.820484 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.820489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:34.820690 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:34.821350 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:34.821376 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:34.823509 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:34.826301 (+ 2792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:34.826308 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:34.826322 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:34.826766 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:34.827090 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:34.827093 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:34.827096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:34.827286 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:34.827820 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:34.827826 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:34.828139 (+ 313us) client_negotiation.cc:769] Sending connection context
0928 14:50:34.828400 (+ 261us) client_negotiation.cc:240] Negotiation successful
0928 14:50:34.828808 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":307,"threads_started":1}
I20260928 14:50:34.831290 18857 heartbeater.cc:347] Connected to a master server at 127.14.241.254:43573
I20260928 14:50:34.831589 18857 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:34.832367 18857 heartbeater.cc:511] Master 127.14.241.254:43573 requested a full tablet report, sending...
I20260928 14:50:34.833657 18401 ts_manager.cc:194] Registered new tserver with Master: 45e65d0c35034617920a02af42872591 (127.14.241.195:36061)
I20260928 14:50:34.834434 18401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:51777
I20260928 14:50:34.844027 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:50:34 dist-test-slave-vk40 krb5kdc[18356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607034, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
I20260928 14:50:34.876116 18590 heartbeater.cc:503] Master 127.14.241.254:43573 was elected leader, sending a full tablet report...
Sep 28 14:50:34 dist-test-slave-vk40 krb5kdc[18356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607034, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:34.889981 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:34.876897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49484 (local address 127.14.241.254:43573)
0928 14:50:34.877083 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:34.877089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:34.877548 (+ 459us) server_negotiation.cc:407] Connection header received
0928 14:50:34.877719 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:34.877725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:34.877804 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:34.877900 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:34.879210 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.879919 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.881137 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.881560 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.881653 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:34.885323 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:34.885329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:34.885332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:34.885368 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:34.887596 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.888165 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.888168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.888171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.888336 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.888696 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.888701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.888704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.889058 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:34.889272 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:34.889562 (+ 290us) server_negotiation.cc:299] Negotiation successful
0928 14:50:34.889736 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260928 14:50:34.890596 18401 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49484
I20260928 14:50:34.892340 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18462
Sep 28 14:50:34 dist-test-slave-vk40 krb5kdc[18356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607034, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:34.907157 18868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:34.893420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.14.241.193:35101)
0928 14:50:34.893974 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:34.893981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:34.894003 (+ 22us) server_negotiation.cc:407] Connection header received
0928 14:50:34.894058 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:34.894064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:34.894275 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:34.894404 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:34.895530 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.896588 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.897769 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.898307 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.898408 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:34.902216 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:34.902228 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:34.902242 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:34.902279 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:34.904833 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.905419 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.905426 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.905428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.905548 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.905805 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.905808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.905810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.906122 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:34.906404 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:34.906595 (+ 191us) server_negotiation.cc:299] Negotiation successful
0928 14:50:34.906811 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":258,"threads_started":1}
W20260928 14:50:34.908466 18585 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40658
W20260928 14:50:34.909503 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18462: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:50:34.909680 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 18462
I20260928 14:50:34.922804 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18596
Sep 28 14:50:34 dist-test-slave-vk40 krb5kdc[18356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607034, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:50:34.937527 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:34.923653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38970 (local address 127.14.241.194:38243)
0928 14:50:34.924207 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:34.924214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:34.924238 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:50:34.924330 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:34.924339 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:34.924552 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:34.924678 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:34.925781 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.926822 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.927973 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.928630 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.928729 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:34.932605 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:34.932618 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:34.932634 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:34.932674 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:34.935162 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.935743 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.935751 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.935753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.935888 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.936157 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.936160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.936162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.936519 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:34.936742 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:34.936966 (+ 224us) server_negotiation.cc:299] Negotiation successful
0928 14:50:34.937162 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":291,"threads_started":1}
W20260928 14:50:34.938704 18719 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38970
W20260928 14:50:34.939762 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18596: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:50:34.939925 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 18596
I20260928 14:50:34.952592 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18729
Sep 28 14:50:34 dist-test-slave-vk40 krb5kdc[18356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607034, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:50:34.968137 18870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:34.953460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41606 (local address 127.14.241.195:36061)
0928 14:50:34.954009 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:34.954016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:34.954048 (+ 32us) server_negotiation.cc:407] Connection header received
0928 14:50:34.954103 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:34.954109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:34.954329 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:34.954448 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:34.955708 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.956824 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.958052 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:34.958635 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:34.958738 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:34.962913 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:34.962924 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:34.962938 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:34.962974 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:34.965688 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.966360 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.966367 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.966370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.966505 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:34.966794 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:34.966797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:34.966799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:34.967147 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:34.967380 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:34.967577 (+ 197us) server_negotiation.cc:299] Negotiation successful
0928 14:50:34.967783 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":244,"threads_started":1}
W20260928 14:50:34.969457 18852 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41606
W20260928 14:50:34.970539 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18729: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:50:34.970701 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 18729
I20260928 14:50:34.983076 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18369
I20260928 14:50:34.984253 18431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:35.093479 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 18369
2026-09-28T14:50:35Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3431 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:50:35 dist-test-slave-vk40 krb5kdc[18874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:50:35 dist-test-slave-vk40 krb5kdc[18874](info): set up 2 sockets
Sep 28 14:50:35 dist-test-slave-vk40 krb5kdc[18874](info): commencing operation
krb5kdc: starting...
W20260928 14:50:36.099232 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:50:36 dist-test-slave-vk40 krb5kdc[18874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:50:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:50:36.287856 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41161
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:41473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:41161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:36.600080 18887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:36.600397 18887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:36.600500 18887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:36.608314 18887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:36.608462 18887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:36.608510 18887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:36.608569 18887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:36.608613 18887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:36.618705 18887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:41473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:41161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:36.619975 18887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:36.621289 18887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:36.633160 18893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:36.633205 18892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:36.633227 18895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:36.633975 18887 server_base.cc:1034] running on GCE node
I20260928 14:50:36.634752 18887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:36.636178 18887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:36.637403 18887 hybrid_clock.cc:648] HybridClock initialized: now 1790607036637342 us; error 39 us; skew 500 ppm
I20260928 14:50:36.637782 18887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:36 dist-test-slave-vk40 krb5kdc[18874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607036, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:36.640970 18887 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:50:36.642454 18887 webserver.cc:460] Webserver started at http://127.14.241.254:39873/ using document root <none> and password file <none>
I20260928 14:50:36.643126 18887 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:36.643239 18887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:36.643570 18887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:36.646575 18887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "ca6d60c9752044848e736d011a2eaa7c"
format_stamp: "Formatted at 2026-09-28 14:50:36 on dist-test-slave-vk40"
server_key: "dd60e44c94e01080ab4e9d61da944d63"
server_key_iv: "d8117da1396ff54128ba080cb3a68759"
server_key_version: "encryptionkey@0"
I20260928 14:50:36.647356 18887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "ca6d60c9752044848e736d011a2eaa7c"
format_stamp: "Formatted at 2026-09-28 14:50:36 on dist-test-slave-vk40"
server_key: "dd60e44c94e01080ab4e9d61da944d63"
server_key_iv: "d8117da1396ff54128ba080cb3a68759"
server_key_version: "encryptionkey@0"
I20260928 14:50:36.652488 18887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260928 14:50:36.655929 18902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:36.657227 18887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:36.657393 18887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "ca6d60c9752044848e736d011a2eaa7c"
format_stamp: "Formatted at 2026-09-28 14:50:36 on dist-test-slave-vk40"
server_key: "dd60e44c94e01080ab4e9d61da944d63"
server_key_iv: "d8117da1396ff54128ba080cb3a68759"
server_key_version: "encryptionkey@0"
I20260928 14:50:36.657567 18887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:36.684737 18887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:36.688968 18887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:36.689227 18887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:36.706228 18887 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:41161
I20260928 14:50:36.706249 18953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:41161 every 8 connection(s)
I20260928 14:50:36.707599 18887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:36.711319 18954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:36.715080 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 18887
I20260928 14:50:36.715282 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:36.715906 15303 external_mini_cluster.cc:1428] Setting key f74ace66beca3aaa8164b74bf0be6749
I20260928 14:50:36.722431 18954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c: Bootstrap starting.
I20260928 14:50:36.726109 18954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:36.727298 18954 log.cc:824] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:36.730540 18954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c: No bootstrap required, opened a new log
Sep 28 14:50:36 dist-test-slave-vk40 krb5kdc[18874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:36.738526 18954 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6d60c9752044848e736d011a2eaa7c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41161 } }
I20260928 14:50:36.738891 18954 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:36.738930 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:36.718013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38542 (local address 127.14.241.254:41161)
0928 14:50:36.718815 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:36.718827 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:36.718870 (+ 43us) server_negotiation.cc:407] Connection header received
0928 14:50:36.720838 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:36.720862 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:36.721295 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:36.721635 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:36.723411 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:36.724874 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:36.726580 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:36.727307 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:36.727462 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:36.731814 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:36.731823 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:36.731839 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:36.731879 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:36.734937 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:36.735604 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:36.735610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:36.735617 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:36.735797 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:36.736167 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:36.736170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:36.736172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:36.736767 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:36.737074 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:36.737508 (+ 434us) server_negotiation.cc:299] Negotiation successful
0928 14:50:36.737924 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":362,"threads_started":1}
I20260928 14:50:36.738986 18954 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6d60c9752044848e736d011a2eaa7c, State: Initialized, Role: FOLLOWER
I20260928 14:50:36.739665 18954 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6d60c9752044848e736d011a2eaa7c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41161 } }
I20260928 14:50:36.739945 18954 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:36.740072 18954 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:36.740255 18954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:36.742730 18954 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6d60c9752044848e736d011a2eaa7c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41161 } }
I20260928 14:50:36.743214 18954 leader_election.cc:302] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca6d60c9752044848e736d011a2eaa7c; no voters:
I20260928 14:50:36.743759 18954 leader_election.cc:288] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:36.743903 18959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:36.744499 18959 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [term 1 LEADER]: Becoming Leader. State: Replica: ca6d60c9752044848e736d011a2eaa7c, State: Running, Role: LEADER
I20260928 14:50:36.745095 18959 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6d60c9752044848e736d011a2eaa7c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41161 } }
I20260928 14:50:36.745659 18954 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:36.747743 18961 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca6d60c9752044848e736d011a2eaa7c. Latest consensus state: current_term: 1 leader_uuid: "ca6d60c9752044848e736d011a2eaa7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6d60c9752044848e736d011a2eaa7c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41161 } } }
I20260928 14:50:36.748075 18961 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:36.748461 18960 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca6d60c9752044848e736d011a2eaa7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6d60c9752044848e736d011a2eaa7c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41161 } } }
I20260928 14:50:36.748834 18960 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:36.757540 18965 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:36.762576 18965 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:36.771030 18965 catalog_manager.cc:1347] Generated new cluster ID: 0590b227cb7046d5b61f3daa6f9b92ec
I20260928 14:50:36.771142 18965 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:36.781118 18965 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:36.782399 18965 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:36.788133 18965 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca6d60c9752044848e736d011a2eaa7c: Generated new TSK 0
I20260928 14:50:36.789188 18965 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:50:36.844651 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41161
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:41473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:37.173990 18980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:37.174309 18980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:37.174443 18980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:37.181733 18980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:37.181866 18980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:37.182008 18980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:37.191754 18980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:41473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:37.193092 18980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:37.194458 18980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:37.207497 18985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:37.207557 18988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:37.207496 18986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:37.208380 18980 server_base.cc:1034] running on GCE node
I20260928 14:50:37.209059 18980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:37.210000 18980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:37.211233 18980 hybrid_clock.cc:648] HybridClock initialized: now 1790607037211177 us; error 46 us; skew 500 ppm
I20260928 14:50:37.211570 18980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:37 dist-test-slave-vk40 krb5kdc[18874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607037, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:37.214757 18980 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:50:37.216198 18980 webserver.cc:460] Webserver started at http://127.14.241.193:42057/ using document root <none> and password file <none>
I20260928 14:50:37.217038 18980 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:37.217146 18980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:37.217427 18980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:37.220366 18980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "666d0d6cac67487f9d5574a35f5fce1c"
format_stamp: "Formatted at 2026-09-28 14:50:37 on dist-test-slave-vk40"
server_key: "3b14542dc3e101ef4d133bca4a9b6a16"
server_key_iv: "73252a03ae36edb321f616b5168f58a0"
server_key_version: "encryptionkey@0"
I20260928 14:50:37.221122 18980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "666d0d6cac67487f9d5574a35f5fce1c"
format_stamp: "Formatted at 2026-09-28 14:50:37 on dist-test-slave-vk40"
server_key: "3b14542dc3e101ef4d133bca4a9b6a16"
server_key_iv: "73252a03ae36edb321f616b5168f58a0"
server_key_version: "encryptionkey@0"
I20260928 14:50:37.226213 18980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:50:37.230031 18995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:37.231397 18980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:50:37.231552 18980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "666d0d6cac67487f9d5574a35f5fce1c"
format_stamp: "Formatted at 2026-09-28 14:50:37 on dist-test-slave-vk40"
server_key: "3b14542dc3e101ef4d133bca4a9b6a16"
server_key_iv: "73252a03ae36edb321f616b5168f58a0"
server_key_version: "encryptionkey@0"
I20260928 14:50:37.231742 18980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:37.245378 18980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:37.249593 18980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:37.249871 18980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:37.250799 18980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:37.252094 18980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:37.252162 18980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:37.252271 18980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:37.252362 18980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260928 14:50:37.283581 18980 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:41825
I20260928 14:50:37.283618 19107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:41825 every 8 connection(s)
I20260928 14:50:37.284909 18980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:37.294723 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 18980
I20260928 14:50:37.294938 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:50:37.295547 15303 external_mini_cluster.cc:1428] Setting key 113e7e07e9cb2bc5673911e060b1403c
Sep 28 14:50:37 dist-test-slave-vk40 krb5kdc[18874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607037, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:37.304844 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:37.287396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:39137 (local address 127.14.241.254:41161)
0928 14:50:37.287800 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:37.287805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:37.288840 (+ 1035us) server_negotiation.cc:407] Connection header received
0928 14:50:37.290652 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:37.290657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:37.290761 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:37.290857 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:37.292391 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:37.293132 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:37.294708 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:37.295240 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:37.295363 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:37.299122 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:37.299127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:37.299131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:37.299165 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:37.301622 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:37.302393 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:37.302398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:37.302400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:37.302526 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:37.302962 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:37.302966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:37.302968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:37.303242 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:37.303383 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:37.304216 (+ 833us) server_negotiation.cc:299] Negotiation successful
0928 14:50:37.304489 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":215,"threads_started":1}
I20260928 14:50:37.305926 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:37.287928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41161 (local address 127.14.241.193:39137)
0928 14:50:37.288686 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:37.288724 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:37.290382 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:37.291111 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:37.291120 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:37.291633 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:37.292217 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:37.292234 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:37.293289 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:37.293293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:37.294559 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:37.294572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:37.295466 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:37.295473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:37.295738 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:37.296406 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:37.296429 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:37.298893 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:37.301771 (+ 2878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:37.301782 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:37.301801 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:37.302260 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:37.302629 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:37.302635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:37.302639 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:37.302849 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:37.303365 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:37.303374 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:37.303740 (+ 366us) client_negotiation.cc:769] Sending connection context
0928 14:50:37.304099 (+ 359us) client_negotiation.cc:240] Negotiation successful
0928 14:50:37.304693 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":301,"threads_started":1}
I20260928 14:50:37.307600 19108 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41161
I20260928 14:50:37.307982 19108 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:37.308959 19108 heartbeater.cc:511] Master 127.14.241.254:41161 requested a full tablet report, sending...
I20260928 14:50:37.311550 18919 ts_manager.cc:194] Registered new tserver with Master: 666d0d6cac67487f9d5574a35f5fce1c (127.14.241.193:41825)
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
I20260928 14:50:37.313658 18919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:39137
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:50:37.339283 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41161
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:41473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:37.653563 19114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:37.653854 19114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:37.653952 19114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:37.661208 19114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:37.661327 19114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:37.661448 19114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:37.671388 19114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:41473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:37.672628 19114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:37.673902 19114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:37.686877 19119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:37.686810 19120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:37.686841 19122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:37.687656 19114 server_base.cc:1034] running on GCE node
I20260928 14:50:37.688378 19114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:37.689375 19114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:37.690611 19114 hybrid_clock.cc:648] HybridClock initialized: now 1790607037690540 us; error 57 us; skew 500 ppm
I20260928 14:50:37.691005 19114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:37 dist-test-slave-vk40 krb5kdc[18874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607037, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:37.694402 19114 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:50:37.695983 19114 webserver.cc:460] Webserver started at http://127.14.241.194:42175/ using document root <none> and password file <none>
I20260928 14:50:37.696800 19114 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:37.696925 19114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:37.697233 19114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:37.700313 19114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "94ab8143c3cb42f5ac6ec26171960609"
format_stamp: "Formatted at 2026-09-28 14:50:37 on dist-test-slave-vk40"
server_key: "5a7e1d195d646f9014f65b45a0b39c41"
server_key_iv: "817f5a8d4b4d0c8072150b2cf73855fc"
server_key_version: "encryptionkey@0"
I20260928 14:50:37.701154 19114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "94ab8143c3cb42f5ac6ec26171960609"
format_stamp: "Formatted at 2026-09-28 14:50:37 on dist-test-slave-vk40"
server_key: "5a7e1d195d646f9014f65b45a0b39c41"
server_key_iv: "817f5a8d4b4d0c8072150b2cf73855fc"
server_key_version: "encryptionkey@0"
I20260928 14:50:37.706753 19114 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:50:37.711335 19129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:37.713037 19114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260928 14:50:37.713209 19114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "94ab8143c3cb42f5ac6ec26171960609"
format_stamp: "Formatted at 2026-09-28 14:50:37 on dist-test-slave-vk40"
server_key: "5a7e1d195d646f9014f65b45a0b39c41"
server_key_iv: "817f5a8d4b4d0c8072150b2cf73855fc"
server_key_version: "encryptionkey@0"
I20260928 14:50:37.713454 19114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:37.731469 19114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:37.735728 19114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:37.735966 19114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:37.737026 19114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:37.738306 19114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:37.738382 19114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:37.738471 19114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:37.738515 19114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:37.767807 19114 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:45389
I20260928 14:50:37.767886 19241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:45389 every 8 connection(s)
I20260928 14:50:37.769163 19114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:37.777319 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 19114
I20260928 14:50:37.777498 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:50:37.778012 15303 external_mini_cluster.cc:1428] Setting key 70543733774e45ba3edc716f8a99b66b
Sep 28 14:50:37 dist-test-slave-vk40 krb5kdc[18874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607037, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:37.791139 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:37.771839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:48085 (local address 127.14.241.254:41161)
0928 14:50:37.772018 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:37.772023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:37.773231 (+ 1208us) server_negotiation.cc:407] Connection header received
0928 14:50:37.775040 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:37.775045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:37.775116 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:37.775207 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:37.776885 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:37.777670 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:37.779633 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:37.780161 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:37.780257 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:37.784463 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:37.784468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:37.784472 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:37.784512 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:37.787604 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:37.788528 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:37.788532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:37.788535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:37.788701 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:37.789155 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:37.789159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:37.789161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:37.789565 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:37.789738 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:37.790535 (+ 797us) server_negotiation.cc:299] Negotiation successful
0928 14:50:37.790791 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 14:50:37.792194 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:37.772309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41161 (local address 127.14.241.194:48085)
0928 14:50:37.773055 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:37.773104 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:37.774753 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:37.775476 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:37.775485 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:37.776029 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:37.776711 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:37.776732 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:37.777826 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:37.777832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:37.779489 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:37.779501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:37.780405 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:37.780413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:37.780710 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:37.781412 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:37.781441 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:37.784128 (+ 2687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:37.787809 (+ 3681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:37.787818 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:37.787833 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:37.788363 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:37.788823 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:37.788828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:37.788832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:37.789033 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:37.789756 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:37.789765 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:37.790146 (+ 381us) client_negotiation.cc:769] Sending connection context
0928 14:50:37.790406 (+ 260us) client_negotiation.cc:240] Negotiation successful
0928 14:50:37.791004 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":273,"threads_started":1}
I20260928 14:50:37.793972 19242 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41161
I20260928 14:50:37.794312 19242 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:37.795071 19242 heartbeater.cc:511] Master 127.14.241.254:41161 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
I20260928 14:50:37.796696 18919 ts_manager.cc:194] Registered new tserver with Master: 94ab8143c3cb42f5ac6ec26171960609 (127.14.241.194:45389)
I20260928 14:50:37.797552 18919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:48085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:50:37.823161 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41161
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:41473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:38.138767 19247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:38.139058 19247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:38.139178 19247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:38.146567 19247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:38.146703 19247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:38.146858 19247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:38.156947 19247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:41473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:38.158221 19247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:38.159540 19247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:38.172525 19253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:38.172585 19252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:38.172564 19255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:38.173899 19247 server_base.cc:1034] running on GCE node
I20260928 14:50:38.174499 19247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:38.175446 19247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:38.176637 19247 hybrid_clock.cc:648] HybridClock initialized: now 1790607038176574 us; error 53 us; skew 500 ppm
I20260928 14:50:38.176956 19247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:38 dist-test-slave-vk40 krb5kdc[18874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607038, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:38.179936 19247 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:50:38.181321 19247 webserver.cc:460] Webserver started at http://127.14.241.195:46293/ using document root <none> and password file <none>
I20260928 14:50:38.181995 19247 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:38.182098 19247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:38.182385 19247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:38.185276 19247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "d814704d44f440c0834c1b8dcd1b1580"
format_stamp: "Formatted at 2026-09-28 14:50:38 on dist-test-slave-vk40"
server_key: "b8f39880331b487f0081168795d3b224"
server_key_iv: "56e482bfa05e218eda9597e49b2b8449"
server_key_version: "encryptionkey@0"
I20260928 14:50:38.186017 19247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "d814704d44f440c0834c1b8dcd1b1580"
format_stamp: "Formatted at 2026-09-28 14:50:38 on dist-test-slave-vk40"
server_key: "b8f39880331b487f0081168795d3b224"
server_key_iv: "56e482bfa05e218eda9597e49b2b8449"
server_key_version: "encryptionkey@0"
I20260928 14:50:38.191104 19247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:50:38.194932 19262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:38.196851 19247 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 14:50:38.197060 19247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "d814704d44f440c0834c1b8dcd1b1580"
format_stamp: "Formatted at 2026-09-28 14:50:38 on dist-test-slave-vk40"
server_key: "b8f39880331b487f0081168795d3b224"
server_key_iv: "56e482bfa05e218eda9597e49b2b8449"
server_key_version: "encryptionkey@0"
I20260928 14:50:38.197299 19247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:38.215945 19247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:38.220665 19247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:38.220927 19247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:38.221902 19247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:38.223246 19247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:38.223341 19247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:38.223436 19247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:38.223480 19247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:38.254087 19247 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:36011
I20260928 14:50:38.254128 19375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:36011 every 8 connection(s)
I20260928 14:50:38.255326 19247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:38.261677 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 19247
I20260928 14:50:38.261883 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:50:38.262269 15303 external_mini_cluster.cc:1428] Setting key 92d9b2aa193162552aab3cadbff9980e
Sep 28 14:50:38 dist-test-slave-vk40 krb5kdc[18874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607038, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:38.275259 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:38.257778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:45217 (local address 127.14.241.254:41161)
0928 14:50:38.258002 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:38.258007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:38.259111 (+ 1104us) server_negotiation.cc:407] Connection header received
0928 14:50:38.260992 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:38.260998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:38.261076 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:38.261164 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:38.262795 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.263632 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.265534 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.265962 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.266057 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:38.269532 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:38.269537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:38.269540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:38.269574 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:38.272128 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.273008 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.273012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.273014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.273145 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.273568 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.273572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.273574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.273912 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:38.274034 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:38.274809 (+ 775us) server_negotiation.cc:299] Negotiation successful
0928 14:50:38.275002 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260928 14:50:38.276361 19379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:38.258262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41161 (local address 127.14.241.195:45217)
0928 14:50:38.258937 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:38.258984 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:38.260693 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:38.261432 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:38.261441 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:38.261992 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:38.262607 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:38.262626 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.263781 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.263785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:38.265410 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:38.265422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.266185 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.266191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:38.266408 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:38.267110 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:38.267138 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:38.269327 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:38.272366 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:38.272375 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:38.272390 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:38.272866 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:38.273237 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:38.273240 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:38.273242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:38.273431 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:38.274031 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:38.274037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:38.274409 (+ 372us) client_negotiation.cc:769] Sending connection context
0928 14:50:38.274642 (+ 233us) client_negotiation.cc:240] Negotiation successful
0928 14:50:38.275094 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":262,"threads_started":1}
I20260928 14:50:38.277845 19377 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41161
I20260928 14:50:38.278177 19377 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:38.278906 19377 heartbeater.cc:511] Master 127.14.241.254:41161 requested a full tablet report, sending...
I20260928 14:50:38.280423 18919 ts_manager.cc:194] Registered new tserver with Master: d814704d44f440c0834c1b8dcd1b1580 (127.14.241.195:36011)
I20260928 14:50:38.281203 18919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:45217
I20260928 14:50:38.289958 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:50:38 dist-test-slave-vk40 krb5kdc[18874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260928 14:50:38.317034 19108 heartbeater.cc:503] Master 127.14.241.254:41161 was elected leader, sending a full tablet report...
Sep 28 14:50:38 dist-test-slave-vk40 krb5kdc[18874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:38.336421 19384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:38.321794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.14.241.193:41825)
0928 14:50:38.322412 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:38.322422 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:38.322447 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:50:38.322504 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:38.322510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:38.322728 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:38.322857 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:38.324214 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.325323 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.326564 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.327154 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.327245 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:38.330970 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:38.330983 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:38.330998 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:38.331042 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:38.333762 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.334408 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.334416 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.334418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.334553 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.334876 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.334880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.334883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.335306 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:38.335543 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:38.335775 (+ 232us) server_negotiation.cc:299] Negotiation successful
0928 14:50:38.336002 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":313,"threads_started":1}
Sep 28 14:50:38 dist-test-slave-vk40 krb5kdc[18874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:38.353111 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:38.340384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38552 (local address 127.14.241.254:41161)
0928 14:50:38.340589 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:38.340595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:38.340731 (+ 136us) server_negotiation.cc:407] Connection header received
0928 14:50:38.340870 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:38.340873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:38.340936 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:38.341004 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:38.342148 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.342877 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.344069 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.344545 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.344649 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:38.348155 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:38.348160 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:38.348163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:38.348201 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:38.350704 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.351395 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.351400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.351402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.351539 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.351838 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.351841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.351843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.352161 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:38.352388 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:38.352627 (+ 239us) server_negotiation.cc:299] Negotiation successful
0928 14:50:38.352818 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260928 14:50:38.353752 18919 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38552
I20260928 14:50:38.354748 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18980
I20260928 14:50:38.367363 19384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:38.355839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34706 (local address 127.14.241.193:41825)
0928 14:50:38.356030 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:38.356036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:38.356170 (+ 134us) server_negotiation.cc:407] Connection header received
0928 14:50:38.356367 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:38.356371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:38.356446 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:38.356526 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:38.357699 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.358375 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.359553 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.359983 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.360096 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:38.362373 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:38.362378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:38.362382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:38.362415 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:38.364885 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.365506 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.365511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.365514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.365638 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.365986 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.365989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.365992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.366244 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:38.366510 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:38.366834 (+ 324us) server_negotiation.cc:299] Negotiation successful
0928 14:50:38.367049 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:50:38.367967 19103 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:38.450591 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 18980
I20260928 14:50:38.464911 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 19114
Sep 28 14:50:38 dist-test-slave-vk40 krb5kdc[18874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:50:38.480739 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:38.465712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.14.241.194:45389)
0928 14:50:38.466257 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:38.466264 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:38.466289 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:50:38.466356 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:38.466362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:38.466583 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:38.466725 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:38.467880 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.468991 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.470285 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.470834 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.470929 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:38.475080 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:38.475093 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:38.475108 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:38.475146 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:38.477940 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.478618 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.478625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.478628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.478750 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.479110 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.479113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.479115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.479462 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:38.479785 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:38.480076 (+ 291us) server_negotiation.cc:299] Negotiation successful
0928 14:50:38.480366 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":300,"threads_started":1}
I20260928 14:50:38.481957 19237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:38.563445 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 19114
I20260928 14:50:38.576781 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 19247
Sep 28 14:50:38 dist-test-slave-vk40 krb5kdc[18874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:50:38.592849 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:38.577771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60404 (local address 127.14.241.195:36011)
0928 14:50:38.578395 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:38.578402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:38.578427 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:50:38.578490 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:38.578496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:38.578691 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:38.578863 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:38.580086 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.581146 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.582439 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:38.583026 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:38.583137 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:38.587432 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:38.587446 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:38.587463 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:38.587503 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:38.590257 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.590944 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.590951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.590953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.591074 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:38.591434 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:38.591436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:38.591438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:38.591812 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:38.592033 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:38.592322 (+ 289us) server_negotiation.cc:299] Negotiation successful
0928 14:50:38.592541 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":336,"threads_started":1}
I20260928 14:50:38.594108 19371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:38.677433 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 19247
I20260928 14:50:38.689800 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 18887
I20260928 14:50:38.690903 18949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:38.793962 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 18887
2026-09-28T14:50:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3699 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:50:38 dist-test-slave-vk40 krb5kdc[19396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:50:38 dist-test-slave-vk40 krb5kdc[19396](info): set up 2 sockets
Sep 28 14:50:38 dist-test-slave-vk40 krb5kdc[19396](info): commencing operation
krb5kdc: starting...
W20260928 14:50:39.791332 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.003s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:50:39 dist-test-slave-vk40 krb5kdc[19396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:50:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:50:39.985560 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46079
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:46079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:40.312254 19409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:40.312570 19409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:40.312691 19409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:40.320441 19409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:40.320565 19409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:40.320636 19409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:40.320715 19409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:40.320780 19409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:40.330778 19409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:46079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:40.332122 19409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:40.333480 19409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:40.344569 19414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:40.344594 19415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:40.344673 19417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:40.345171 19409 server_base.cc:1034] running on GCE node
I20260928 14:50:40.345985 19409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:40.347440 19409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:40.348628 19409 hybrid_clock.cc:648] HybridClock initialized: now 1790607040348572 us; error 55 us; skew 500 ppm
I20260928 14:50:40.348970 19409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:40 dist-test-slave-vk40 krb5kdc[19396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607040, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:40.358946 19409 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:50:40.360232 19409 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:40.360396 19409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:40.360742 19409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:40.363903 19409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "68bc8c9807f24fbb9fe055ebf379d11f"
format_stamp: "Formatted at 2026-09-28 14:50:40 on dist-test-slave-vk40"
server_key: "aa66e261742b1752023c62c719401bc4"
server_key_iv: "cc375b551597a052d419766b31162f97"
server_key_version: "encryptionkey@0"
I20260928 14:50:40.364753 19409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "68bc8c9807f24fbb9fe055ebf379d11f"
format_stamp: "Formatted at 2026-09-28 14:50:40 on dist-test-slave-vk40"
server_key: "aa66e261742b1752023c62c719401bc4"
server_key_iv: "cc375b551597a052d419766b31162f97"
server_key_version: "encryptionkey@0"
I20260928 14:50:40.370231 19409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260928 14:50:40.373983 19423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:40.375396 19409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:50:40.375562 19409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "68bc8c9807f24fbb9fe055ebf379d11f"
format_stamp: "Formatted at 2026-09-28 14:50:40 on dist-test-slave-vk40"
server_key: "aa66e261742b1752023c62c719401bc4"
server_key_iv: "cc375b551597a052d419766b31162f97"
server_key_version: "encryptionkey@0"
I20260928 14:50:40.375771 19409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:40.402241 19409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:40.406371 19409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:40.406636 19409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:40.424218 19409 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:46079
I20260928 14:50:40.424240 19475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:46079 every 8 connection(s)
I20260928 14:50:40.425149 19409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:40.428866 19476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:40.434158 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 19409
I20260928 14:50:40.434397 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:40.435158 15303 external_mini_cluster.cc:1428] Setting key 804cc84b5e013d78281648ed336a31ee
I20260928 14:50:40.440868 19476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f: Bootstrap starting.
I20260928 14:50:40.444722 19476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:40.446128 19476 log.cc:824] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:40.449807 19476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f: No bootstrap required, opened a new log
Sep 28 14:50:40 dist-test-slave-vk40 krb5kdc[19396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:40.459184 19476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bc8c9807f24fbb9fe055ebf379d11f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46079 } }
I20260928 14:50:40.459595 19476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:40.459641 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:40.437312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54742 (local address 127.14.241.254:46079)
0928 14:50:40.438158 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:40.438170 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:40.438222 (+ 52us) server_negotiation.cc:407] Connection header received
0928 14:50:40.439996 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:40.440020 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:40.440550 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:40.440909 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:40.442804 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:40.444177 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:40.446076 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:40.447097 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:40.447363 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:40.451951 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:40.451960 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:40.451977 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:40.452027 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:40.455212 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:40.456104 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:40.456112 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:40.456119 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:40.456347 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:40.456744 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:40.456747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:40.456748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:40.457364 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:40.457605 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:40.458081 (+ 476us) server_negotiation.cc:299] Negotiation successful
0928 14:50:40.458499 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":362,"threads_started":1}
I20260928 14:50:40.459697 19476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68bc8c9807f24fbb9fe055ebf379d11f, State: Initialized, Role: FOLLOWER
I20260928 14:50:40.460460 19476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bc8c9807f24fbb9fe055ebf379d11f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46079 } }
I20260928 14:50:40.460758 19476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:40.460894 19476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:40.461072 19476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:40.463696 19476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bc8c9807f24fbb9fe055ebf379d11f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46079 } }
I20260928 14:50:40.464198 19476 leader_election.cc:302] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68bc8c9807f24fbb9fe055ebf379d11f; no voters:
I20260928 14:50:40.464787 19476 leader_election.cc:288] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:40.464942 19481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:40.465590 19481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [term 1 LEADER]: Becoming Leader. State: Replica: 68bc8c9807f24fbb9fe055ebf379d11f, State: Running, Role: LEADER
I20260928 14:50:40.466153 19481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bc8c9807f24fbb9fe055ebf379d11f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46079 } }
I20260928 14:50:40.466888 19476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:40.469013 19483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68bc8c9807f24fbb9fe055ebf379d11f. Latest consensus state: current_term: 1 leader_uuid: "68bc8c9807f24fbb9fe055ebf379d11f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bc8c9807f24fbb9fe055ebf379d11f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46079 } } }
I20260928 14:50:40.469313 19483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:40.469739 19482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68bc8c9807f24fbb9fe055ebf379d11f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bc8c9807f24fbb9fe055ebf379d11f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46079 } } }
I20260928 14:50:40.470067 19482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:40.471009 19488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:40.476416 19488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:40.486032 19488 catalog_manager.cc:1347] Generated new cluster ID: eee16e2b884f4cf6900d301887a78e77
I20260928 14:50:40.486245 19488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:40.508493 19488 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:40.510494 19488 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:40.521768 19488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f: Generated new TSK 0
I20260928 14:50:40.522986 19488 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:50:40.584074 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46079
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:40.915493 19502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:40.915792 19502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:40.915926 19502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:40.923360 19502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:40.923478 19502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:40.923628 19502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:40.933559 19502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46079
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:40.934861 19502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:40.936120 19502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:40.948807 19507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:40.948890 19510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:40.948894 19508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:40.949942 19502 server_base.cc:1034] running on GCE node
I20260928 14:50:40.950516 19502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:40.951495 19502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:40.952680 19502 hybrid_clock.cc:648] HybridClock initialized: now 1790607040952620 us; error 54 us; skew 500 ppm
I20260928 14:50:40.952999 19502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:40 dist-test-slave-vk40 krb5kdc[19396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607040, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:40.956092 19502 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:50:40.957283 19502 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:40.957376 19502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:40.957686 19502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:40.960676 19502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "0f6a97bd327a44b78335c8380e15893c"
format_stamp: "Formatted at 2026-09-28 14:50:40 on dist-test-slave-vk40"
server_key: "1c97db036b4483941a734b04a9768be6"
server_key_iv: "9fac857a96d973441f7b767a7ec4e3af"
server_key_version: "encryptionkey@0"
I20260928 14:50:40.961442 19502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "0f6a97bd327a44b78335c8380e15893c"
format_stamp: "Formatted at 2026-09-28 14:50:40 on dist-test-slave-vk40"
server_key: "1c97db036b4483941a734b04a9768be6"
server_key_iv: "9fac857a96d973441f7b767a7ec4e3af"
server_key_version: "encryptionkey@0"
I20260928 14:50:40.966470 19502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260928 14:50:40.970077 19516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:40.971561 19502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:50:40.971714 19502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "0f6a97bd327a44b78335c8380e15893c"
format_stamp: "Formatted at 2026-09-28 14:50:40 on dist-test-slave-vk40"
server_key: "1c97db036b4483941a734b04a9768be6"
server_key_iv: "9fac857a96d973441f7b767a7ec4e3af"
server_key_version: "encryptionkey@0"
I20260928 14:50:40.971953 19502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:40.998401 19502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:41.002736 19502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:41.003011 19502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:41.003983 19502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:41.005357 19502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:41.005441 19502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:41.005537 19502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:41.005606 19502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:41.035903 19502 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:44081
I20260928 14:50:41.035955 19629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:44081 every 8 connection(s)
I20260928 14:50:41.036799 19502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:41.043074 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 19502
I20260928 14:50:41.043301 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:50:41.044063 15303 external_mini_cluster.cc:1428] Setting key 36bdf129416ea9be3059612e835ca1cc
Sep 28 14:50:41 dist-test-slave-vk40 krb5kdc[19396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607040, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:41.058104 19632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:41.039074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:36205 (local address 127.14.241.254:46079)
0928 14:50:41.039482 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:41.039486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:41.040434 (+ 948us) server_negotiation.cc:407] Connection header received
0928 14:50:41.042288 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:41.042294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:41.042389 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:41.042494 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:41.044767 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:41.045749 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:41.047521 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:41.047929 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:41.048040 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:41.051771 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:41.051778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:41.051783 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:41.051822 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:41.055009 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:41.055792 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:41.055797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:41.055801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:41.055925 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:41.056358 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:41.056363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:41.056366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:41.056634 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:41.056765 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:41.057487 (+ 722us) server_negotiation.cc:299] Negotiation successful
0928 14:50:41.057720 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":202,"threads_started":1}
I20260928 14:50:41.059165 19633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:41.039542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46079 (local address 127.14.241.193:36205)
0928 14:50:41.040231 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:41.040269 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:41.041993 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:41.042808 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:41.042821 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:41.043620 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:41.044519 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:41.044548 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:41.045968 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:41.045973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:41.047363 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:41.047377 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:41.048160 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:41.048167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:41.048443 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:41.049095 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:41.049118 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:41.051463 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:41.055171 (+ 3708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:41.055181 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:41.055200 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:41.055648 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:41.056031 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:41.056036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:41.056040 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:41.056224 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:41.056758 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:41.056767 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:41.057119 (+ 352us) client_negotiation.cc:769] Sending connection context
0928 14:50:41.057379 (+ 260us) client_negotiation.cc:240] Negotiation successful
0928 14:50:41.057936 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":292,"threads_started":1}
I20260928 14:50:41.060778 19630 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46079
I20260928 14:50:41.061066 19630 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:41.061712 19630 heartbeater.cc:511] Master 127.14.241.254:46079 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
I20260928 14:50:41.063745 19440 ts_manager.cc:194] Registered new tserver with Master: 0f6a97bd327a44b78335c8380e15893c (127.14.241.193:44081)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
I20260928 14:50:41.065796 19440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:36205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:50:41.096874 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46079
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:41.423693 19636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:41.423971 19636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:41.424096 19636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:41.431596 19636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:41.431712 19636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:41.431876 19636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:41.441942 19636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46079
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:41.443298 19636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:41.444612 19636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:41.457341 19644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:41.457376 19642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:41.457296 19641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:41.458285 19636 server_base.cc:1034] running on GCE node
I20260928 14:50:41.458942 19636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:41.459945 19636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:41.461164 19636 hybrid_clock.cc:648] HybridClock initialized: now 1790607041461116 us; error 43 us; skew 500 ppm
I20260928 14:50:41.461500 19636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:41 dist-test-slave-vk40 krb5kdc[19396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607041, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:41.464805 19636 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:50:41.465998 19636 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:41.466105 19636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:41.466413 19636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:41.469698 19636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "73f68f45aa974e06956895264e912fbf"
format_stamp: "Formatted at 2026-09-28 14:50:41 on dist-test-slave-vk40"
server_key: "c21d03de72505d17e2d808eb633185d0"
server_key_iv: "c8fd6817eb64e2e6d1567f4a93e0a4fe"
server_key_version: "encryptionkey@0"
I20260928 14:50:41.470508 19636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "73f68f45aa974e06956895264e912fbf"
format_stamp: "Formatted at 2026-09-28 14:50:41 on dist-test-slave-vk40"
server_key: "c21d03de72505d17e2d808eb633185d0"
server_key_iv: "c8fd6817eb64e2e6d1567f4a93e0a4fe"
server_key_version: "encryptionkey@0"
I20260928 14:50:41.475790 19636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260928 14:50:41.479394 19650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:41.480870 19636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:41.481040 19636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "73f68f45aa974e06956895264e912fbf"
format_stamp: "Formatted at 2026-09-28 14:50:41 on dist-test-slave-vk40"
server_key: "c21d03de72505d17e2d808eb633185d0"
server_key_iv: "c8fd6817eb64e2e6d1567f4a93e0a4fe"
server_key_version: "encryptionkey@0"
I20260928 14:50:41.481300 19636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:41.499823 19636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:41.504196 19636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:41.504467 19636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:41.505445 19636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:41.506817 19636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:41.506902 19636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:41.506991 19636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:41.507046 19636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:41.537659 19636 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:42103
I20260928 14:50:41.537719 19763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:42103 every 8 connection(s)
I20260928 14:50:41.538511 19636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:41.548393 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 19636
I20260928 14:50:41.548570 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:50:41.548982 15303 external_mini_cluster.cc:1428] Setting key e83729f4587a773dc8f222c1491baffa
Sep 28 14:50:41 dist-test-slave-vk40 krb5kdc[19396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607041, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:41.558089 19632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:41.540807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:43039 (local address 127.14.241.254:46079)
0928 14:50:41.541021 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:41.541028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:41.542244 (+ 1216us) server_negotiation.cc:407] Connection header received
0928 14:50:41.544080 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:41.544087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:41.544174 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:41.544319 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:41.545773 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:41.546472 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:41.547969 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:41.548450 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:41.548561 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:41.552435 (+ 3874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:41.552440 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:41.552443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:41.552476 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:41.554943 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:41.555776 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:41.555779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:41.555782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:41.555907 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:41.556377 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:41.556383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:41.556386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:41.556727 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:41.556878 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:41.557537 (+ 659us) server_negotiation.cc:299] Negotiation successful
0928 14:50:41.557753 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 14:50:41.558930 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:41.541282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46079 (local address 127.14.241.194:43039)
0928 14:50:41.542079 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:41.542124 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:41.543765 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:41.544515 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:41.544523 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:41.545055 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:41.545626 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:41.545644 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:41.546594 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:41.546598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:41.547831 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:41.547843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:41.548674 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:41.548683 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:41.548891 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:41.549550 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:41.549573 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:41.552151 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:41.555081 (+ 2930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:41.555091 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:41.555108 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:41.555680 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:41.555997 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:41.556002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:41.556005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:41.556234 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:41.556845 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:41.556850 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:41.557174 (+ 324us) client_negotiation.cc:769] Sending connection context
0928 14:50:41.557398 (+ 224us) client_negotiation.cc:240] Negotiation successful
0928 14:50:41.557808 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":314,"threads_started":1}
I20260928 14:50:41.560508 19764 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46079
I20260928 14:50:41.560796 19764 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:41.561424 19764 heartbeater.cc:511] Master 127.14.241.254:46079 requested a full tablet report, sending...
I20260928 14:50:41.562878 19440 ts_manager.cc:194] Registered new tserver with Master: 73f68f45aa974e06956895264e912fbf (127.14.241.194:42103)
I20260928 14:50:41.563951 19440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:43039
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:50:41.591571 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46079
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:41.905225 19769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:41.905515 19769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:41.905640 19769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:41.913143 19769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:41.913275 19769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:41.913451 19769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:41.923282 19769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46079
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:41.924574 19769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:41.925870 19769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:41.938052 19777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:41.938063 19775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:41.938514 19769 server_base.cc:1034] running on GCE node
W20260928 14:50:41.938651 19774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:41.939298 19769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:41.940156 19769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:41.941363 19769 hybrid_clock.cc:648] HybridClock initialized: now 1790607041941321 us; error 69 us; skew 500 ppm
I20260928 14:50:41.941646 19769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:41 dist-test-slave-vk40 krb5kdc[19396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607041, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:41.944607 19769 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:50:41.945683 19769 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:41.945771 19769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:41.946028 19769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:41.948881 19769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "bed9c021e3244db0990714ae72dda0c8"
format_stamp: "Formatted at 2026-09-28 14:50:41 on dist-test-slave-vk40"
server_key: "6870ec520c6c90fba025672d8cf681fd"
server_key_iv: "554fb600d509fbf25a08c2181d05fd34"
server_key_version: "encryptionkey@0"
I20260928 14:50:41.949600 19769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "bed9c021e3244db0990714ae72dda0c8"
format_stamp: "Formatted at 2026-09-28 14:50:41 on dist-test-slave-vk40"
server_key: "6870ec520c6c90fba025672d8cf681fd"
server_key_iv: "554fb600d509fbf25a08c2181d05fd34"
server_key_version: "encryptionkey@0"
I20260928 14:50:41.954386 19769 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 14:50:41.957649 19783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:41.958957 19769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:50:41.959080 19769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "bed9c021e3244db0990714ae72dda0c8"
format_stamp: "Formatted at 2026-09-28 14:50:41 on dist-test-slave-vk40"
server_key: "6870ec520c6c90fba025672d8cf681fd"
server_key_iv: "554fb600d509fbf25a08c2181d05fd34"
server_key_version: "encryptionkey@0"
I20260928 14:50:41.959262 19769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:41.970389 19769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:41.974407 19769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:41.974625 19769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:41.975481 19769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:41.976828 19769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:41.976909 19769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:41.976984 19769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:41.977033 19769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:42.005421 19769 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:37175
I20260928 14:50:42.005475 19895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:37175 every 8 connection(s)
I20260928 14:50:42.006229 19769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:42.009027 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 19769
I20260928 14:50:42.009223 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:50:42.009788 15303 external_mini_cluster.cc:1428] Setting key 425ac6782646bad18a0f4d07a6dcabd7
Sep 28 14:50:42 dist-test-slave-vk40 krb5kdc[19396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607041, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:42.024833 19632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.008551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:58103 (local address 127.14.241.254:46079)
0928 14:50:42.008722 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:42.008726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:42.010060 (+ 1334us) server_negotiation.cc:407] Connection header received
0928 14:50:42.012010 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:42.012015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:42.012133 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:42.012215 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:42.013767 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.014515 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.016015 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.016444 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.016551 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.019738 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:42.019743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:42.019746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:42.019777 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:42.022028 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:42.022696 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:42.022701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:42.022703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:42.022820 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:42.023193 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:42.023197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:42.023201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:42.023474 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:42.023604 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:42.024252 (+ 648us) server_negotiation.cc:299] Negotiation successful
0928 14:50:42.024498 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 14:50:42.025815 19898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.009098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46079 (local address 127.14.241.195:58103)
0928 14:50:42.009892 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:42.009951 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:42.011761 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:42.012487 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:42.012496 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:42.013057 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:42.013617 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.013633 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.014652 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.014655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.015874 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.015887 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.016645 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.016651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.016865 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.017469 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:42.017490 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:42.019543 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:42.022147 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:42.022157 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:42.022174 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:42.022588 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:42.022917 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:42.022922 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:42.022926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:42.023094 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:42.023591 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:42.023598 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:42.023920 (+ 322us) client_negotiation.cc:769] Sending connection context
0928 14:50:42.024161 (+ 241us) client_negotiation.cc:240] Negotiation successful
0928 14:50:42.024695 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":325,"threads_started":1}
I20260928 14:50:42.027063 19896 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46079
I20260928 14:50:42.027359 19896 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:42.027994 19896 heartbeater.cc:511] Master 127.14.241.254:46079 requested a full tablet report, sending...
I20260928 14:50:42.029424 19440 ts_manager.cc:194] Registered new tserver with Master: bed9c021e3244db0990714ae72dda0c8 (127.14.241.195:37175)
I20260928 14:50:42.030169 19440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:58103
I20260928 14:50:42.036971 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:50:42.052870 19632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.041523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54750 (local address 127.14.241.254:46079)
0928 14:50:42.041745 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:42.041752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:42.042094 (+ 342us) server_negotiation.cc:407] Connection header received
0928 14:50:42.042222 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:42.042226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:42.042307 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:42.042389 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:42.043684 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.044456 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.045642 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.046075 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.046168 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.048199 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:42.048204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:42.048207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:42.048238 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:42.050631 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:42.051277 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:42.051280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:42.051282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:42.051409 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:42.051676 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:42.051679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:42.051681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:42.052005 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:42.052184 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:42.052467 (+ 283us) server_negotiation.cc:299] Negotiation successful
0928 14:50:42.052626 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 14:50:42.057276 19440 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:50:42.064364 19440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:42.069967 19630 heartbeater.cc:503] Master 127.14.241.254:46079 was elected leader, sending a full tablet report...
I20260928 14:50:42.088614 19911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.077248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46820 (local address 127.14.241.194:42103)
0928 14:50:42.079574 (+ 2326us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:42.079584 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:42.079613 (+ 29us) server_negotiation.cc:407] Connection header received
0928 14:50:42.079712 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:42.079720 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:42.080023 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:42.080940 (+ 917us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:42.081168 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.082669 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.085772 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.086962 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.087110 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.087291 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:42.088110 (+ 819us) server_negotiation.cc:299] Negotiation successful
0928 14:50:42.088316 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2127,"thread_start_us":2023,"threads_started":1}
I20260928 14:50:42.088635 19910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.076380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:44081 (local address 127.0.0.1:34472)
0928 14:50:42.077688 (+ 1308us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:42.077724 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:42.077880 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:42.081073 (+ 3193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:42.081078 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:42.081107 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:42.081471 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.081481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.083075 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.083079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.085606 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.085617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.087560 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.087567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.087784 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.087846 (+ 62us) client_negotiation.cc:769] Sending connection context
0928 14:50:42.087955 (+ 109us) client_negotiation.cc:240] Negotiation successful
0928 14:50:42.088171 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"mutex_wait_us":148,"thread_start_us":1786,"threads_started":1}
I20260928 14:50:42.088860 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.076643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34472 (local address 127.14.241.193:44081)
0928 14:50:42.080475 (+ 3832us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:42.080484 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:42.080512 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:50:42.080601 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:42.080608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:42.080856 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:42.081035 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:42.081627 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.082926 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.085760 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.086997 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.087172 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.087347 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:42.088394 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0928 14:50:42.088557 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3663,"thread_start_us":3492,"threads_started":1}
I20260928 14:50:42.088905 19912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.078631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:37175 (local address 127.0.0.1:44342)
0928 14:50:42.079793 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:42.079807 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:42.079907 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:42.082220 (+ 2313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:42.082225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:42.082249 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:42.082579 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.082590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.084404 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.084409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.086308 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.086322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.088075 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.088081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.088230 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.088258 (+ 28us) client_negotiation.cc:769] Sending connection context
0928 14:50:42.088353 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 14:50:42.088478 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":65,"thread_start_us":334,"threads_started":1}
I20260928 14:50:42.089597 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.076729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:42103 (local address 127.0.0.1:46820)
0928 14:50:42.078415 (+ 1686us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:42.078432 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:42.078537 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:42.080352 (+ 1815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:42.080360 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:42.080391 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:42.080758 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.080770 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.082790 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.082795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.085544 (+ 2749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.085563 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.087517 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.087526 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.087776 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.087836 (+ 60us) client_negotiation.cc:769] Sending connection context
0928 14:50:42.087965 (+ 129us) client_negotiation.cc:240] Negotiation successful
0928 14:50:42.089231 (+ 1266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1564,"mutex_wait_us":123,"thread_start_us":591,"threads_started":1}
I20260928 14:50:42.089756 19913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.079082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44342 (local address 127.14.241.195:37175)
0928 14:50:42.081669 (+ 2587us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:42.081677 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:42.081707 (+ 30us) server_negotiation.cc:407] Connection header received
0928 14:50:42.081796 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:42.081803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:42.082031 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:42.082221 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:42.082764 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.084251 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.086483 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.087850 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.089213 (+ 1363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.089333 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:42.089384 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 14:50:42.089503 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2415,"thread_start_us":2263,"threads_started":1}
I20260928 14:50:42.092653 19564 tablet_service.cc:1467] Processing CreateTablet for tablet 9c255edff8f04423ab0f106c9a83541c (DEFAULT_TABLE table=test-table [id=775b7e599f8444fda42c4a895ae88f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:42.093273 19831 tablet_service.cc:1467] Processing CreateTablet for tablet 9c255edff8f04423ab0f106c9a83541c (DEFAULT_TABLE table=test-table [id=775b7e599f8444fda42c4a895ae88f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:42.093273 19698 tablet_service.cc:1467] Processing CreateTablet for tablet 9c255edff8f04423ab0f106c9a83541c (DEFAULT_TABLE table=test-table [id=775b7e599f8444fda42c4a895ae88f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:42.093860 19564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c255edff8f04423ab0f106c9a83541c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:42.094604 19698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c255edff8f04423ab0f106c9a83541c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:42.094609 19831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c255edff8f04423ab0f106c9a83541c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:42.105188 19914 tablet_bootstrap.cc:492] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: Bootstrap starting.
I20260928 14:50:42.108321 19914 tablet_bootstrap.cc:654] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:42.108909 19916 tablet_bootstrap.cc:492] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: Bootstrap starting.
I20260928 14:50:42.109668 19915 tablet_bootstrap.cc:492] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: Bootstrap starting.
I20260928 14:50:42.110011 19914 log.cc:824] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:42.112986 19914 tablet_bootstrap.cc:492] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: No bootstrap required, opened a new log
I20260928 14:50:42.113211 19915 tablet_bootstrap.cc:654] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:42.113258 19914 ts_tablet_manager.cc:1397] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260928 14:50:42.114184 19916 tablet_bootstrap.cc:654] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:42.114425 19915 log.cc:824] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:42.115301 19916 log.cc:824] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:42.117287 19915 tablet_bootstrap.cc:492] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: No bootstrap required, opened a new log
I20260928 14:50:42.117559 19915 ts_tablet_manager.cc:1397] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260928 14:50:42.118080 19916 tablet_bootstrap.cc:492] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: No bootstrap required, opened a new log
I20260928 14:50:42.118325 19916 ts_tablet_manager.cc:1397] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260928 14:50:42.121209 19914 raft_consensus.cc:348] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f6a97bd327a44b78335c8380e15893c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44081 } } peers { permanent_uuid: "73f68f45aa974e06956895264e912fbf" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42103 } } peers { permanent_uuid: "bed9c021e3244db0990714ae72dda0c8" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 37175 } }
I20260928 14:50:42.121589 19914 raft_consensus.cc:374] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:42.121704 19914 raft_consensus.cc:729] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f6a97bd327a44b78335c8380e15893c, State: Initialized, Role: FOLLOWER
I20260928 14:50:42.122299 19914 consensus_queue.cc:260] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f6a97bd327a44b78335c8380e15893c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44081 } } peers { permanent_uuid: "73f68f45aa974e06956895264e912fbf" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42103 } } peers { permanent_uuid: "bed9c021e3244db0990714ae72dda0c8" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 37175 } }
I20260928 14:50:42.124033 19914 ts_tablet_manager.cc:1428] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260928 14:50:42.127900 19915 raft_consensus.cc:348] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f6a97bd327a44b78335c8380e15893c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44081 } } peers { permanent_uuid: "73f68f45aa974e06956895264e912fbf" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42103 } } peers { permanent_uuid: "bed9c021e3244db0990714ae72dda0c8" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 37175 } }
I20260928 14:50:42.128396 19915 raft_consensus.cc:374] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:42.128532 19915 raft_consensus.cc:729] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f68f45aa974e06956895264e912fbf, State: Initialized, Role: FOLLOWER
I20260928 14:50:42.128433 19916 raft_consensus.cc:348] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f6a97bd327a44b78335c8380e15893c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44081 } } peers { permanent_uuid: "73f68f45aa974e06956895264e912fbf" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42103 } } peers { permanent_uuid: "bed9c021e3244db0990714ae72dda0c8" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 37175 } }
I20260928 14:50:42.128925 19916 raft_consensus.cc:374] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:42.129024 19916 raft_consensus.cc:729] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed9c021e3244db0990714ae72dda0c8, State: Initialized, Role: FOLLOWER
I20260928 14:50:42.129280 19915 consensus_queue.cc:260] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f6a97bd327a44b78335c8380e15893c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44081 } } peers { permanent_uuid: "73f68f45aa974e06956895264e912fbf" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42103 } } peers { permanent_uuid: "bed9c021e3244db0990714ae72dda0c8" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 37175 } }
I20260928 14:50:42.129597 19916 consensus_queue.cc:260] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f6a97bd327a44b78335c8380e15893c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44081 } } peers { permanent_uuid: "73f68f45aa974e06956895264e912fbf" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42103 } } peers { permanent_uuid: "bed9c021e3244db0990714ae72dda0c8" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 37175 } }
I20260928 14:50:42.130784 19896 heartbeater.cc:503] Master 127.14.241.254:46079 was elected leader, sending a full tablet report...
I20260928 14:50:42.131503 19916 ts_tablet_manager.cc:1428] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260928 14:50:42.132471 19764 heartbeater.cc:503] Master 127.14.241.254:46079 was elected leader, sending a full tablet report...
I20260928 14:50:42.133108 19915 ts_tablet_manager.cc:1428] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260928 14:50:42.255702 19921 raft_consensus.cc:482] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:42.256028 19921 raft_consensus.cc:504] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f6a97bd327a44b78335c8380e15893c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44081 } } peers { permanent_uuid: "73f68f45aa974e06956895264e912fbf" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42103 } } peers { permanent_uuid: "bed9c021e3244db0990714ae72dda0c8" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 37175 } }
W20260928 14:50:42.257629 19897 tablet.cc:2369] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:50:42.257758 19921 leader_election.cc:288] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f6a97bd327a44b78335c8380e15893c (127.14.241.193:44081), bed9c021e3244db0990714ae72dda0c8 (127.14.241.195:37175)
I20260928 14:50:42.264444 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.258323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:52405 (local address 127.14.241.193:44081)
0928 14:50:42.258696 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:42.258703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:42.259224 (+ 521us) server_negotiation.cc:407] Connection header received
0928 14:50:42.259412 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:42.259416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:42.259475 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:42.259547 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:42.260095 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.260843 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.262658 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.263446 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.263605 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.263685 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:42.264030 (+ 345us) server_negotiation.cc:299] Negotiation successful
0928 14:50:42.264158 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260928 14:50:42.264532 19913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.258792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:51559 (local address 127.14.241.195:37175)
0928 14:50:42.258990 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:42.258996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:42.259308 (+ 312us) server_negotiation.cc:407] Connection header received
0928 14:50:42.259468 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:42.259471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:42.259528 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:42.259598 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:42.260186 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.260929 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.262745 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.263582 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.263730 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.263804 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:42.264138 (+ 334us) server_negotiation.cc:299] Negotiation successful
0928 14:50:42.264263 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:50:42.264578 19923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.258213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:44081 (local address 127.14.241.194:52405)
0928 14:50:42.259107 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:42.259124 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:42.259276 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:42.259574 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:42.259578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:42.259598 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:42.259960 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.259970 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.260958 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.260963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.262504 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.262516 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.263638 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.263644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.263870 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.263903 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 14:50:42.263993 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 14:50:42.264120 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":53,"thread_start_us":539,"threads_started":1}
I20260928 14:50:42.264686 19924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.258283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:37175 (local address 127.14.241.194:51559)
0928 14:50:42.259200 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:42.259217 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:42.259351 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:42.259623 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:42.259626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:42.259652 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:42.260051 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.260059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.261058 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.261062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.262541 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:42.262553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.263858 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.263863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:42.263996 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.264019 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 14:50:42.264096 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:50:42.264242 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":4,"thread_start_us":693,"threads_started":1}
I20260928 14:50:42.265686 19851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c255edff8f04423ab0f106c9a83541c" candidate_uuid: "73f68f45aa974e06956895264e912fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed9c021e3244db0990714ae72dda0c8" is_pre_election: true
I20260928 14:50:42.265686 19584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c255edff8f04423ab0f106c9a83541c" candidate_uuid: "73f68f45aa974e06956895264e912fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f6a97bd327a44b78335c8380e15893c" is_pre_election: true
I20260928 14:50:42.266160 19851 raft_consensus.cc:2454] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73f68f45aa974e06956895264e912fbf in term 0.
I20260928 14:50:42.266161 19584 raft_consensus.cc:2454] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73f68f45aa974e06956895264e912fbf in term 0.
I20260928 14:50:42.266770 19652 leader_election.cc:302] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73f68f45aa974e06956895264e912fbf, bed9c021e3244db0990714ae72dda0c8; no voters:
I20260928 14:50:42.267159 19921 raft_consensus.cc:2781] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:50:42.267282 19921 raft_consensus.cc:482] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:50:42.267374 19921 raft_consensus.cc:3037] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:42.269583 19921 raft_consensus.cc:504] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f6a97bd327a44b78335c8380e15893c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44081 } } peers { permanent_uuid: "73f68f45aa974e06956895264e912fbf" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42103 } } peers { permanent_uuid: "bed9c021e3244db0990714ae72dda0c8" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 37175 } }
I20260928 14:50:42.270290 19921 leader_election.cc:288] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [CANDIDATE]: Term 1 election: Requested vote from peers 0f6a97bd327a44b78335c8380e15893c (127.14.241.193:44081), bed9c021e3244db0990714ae72dda0c8 (127.14.241.195:37175)
I20260928 14:50:42.270617 19584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c255edff8f04423ab0f106c9a83541c" candidate_uuid: "73f68f45aa974e06956895264e912fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f6a97bd327a44b78335c8380e15893c"
I20260928 14:50:42.270792 19851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c255edff8f04423ab0f106c9a83541c" candidate_uuid: "73f68f45aa974e06956895264e912fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed9c021e3244db0990714ae72dda0c8"
I20260928 14:50:42.270825 19584 raft_consensus.cc:3037] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:42.271001 19851 raft_consensus.cc:3037] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:42.274067 19584 raft_consensus.cc:2454] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73f68f45aa974e06956895264e912fbf in term 1.
I20260928 14:50:42.274067 19851 raft_consensus.cc:2454] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73f68f45aa974e06956895264e912fbf in term 1.
I20260928 14:50:42.274578 19654 leader_election.cc:302] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f6a97bd327a44b78335c8380e15893c, 73f68f45aa974e06956895264e912fbf, bed9c021e3244db0990714ae72dda0c8; no voters:
I20260928 14:50:42.274959 19921 raft_consensus.cc:2781] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:42.275405 19921 raft_consensus.cc:686] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 1 LEADER]: Becoming Leader. State: Replica: 73f68f45aa974e06956895264e912fbf, State: Running, Role: LEADER
I20260928 14:50:42.275866 19921 consensus_queue.cc:237] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f6a97bd327a44b78335c8380e15893c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44081 } } peers { permanent_uuid: "73f68f45aa974e06956895264e912fbf" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42103 } } peers { permanent_uuid: "bed9c021e3244db0990714ae72dda0c8" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 37175 } }
I20260928 14:50:42.280628 19440 catalog_manager.cc:5495] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf reported cstate change: term changed from 0 to 1, leader changed from <none> to 73f68f45aa974e06956895264e912fbf (127.14.241.194). New cstate: current_term: 1 leader_uuid: "73f68f45aa974e06956895264e912fbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f6a97bd327a44b78335c8380e15893c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73f68f45aa974e06956895264e912fbf" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bed9c021e3244db0990714ae72dda0c8" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 37175 } health_report { overall_health: UNKNOWN } } }
W20260928 14:50:42.288934 19631 tablet.cc:2369] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:50:42.290184 19765 tablet.cc:2369] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:50:42.328624 19911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.323322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46826 (local address 127.14.241.194:42103)
0928 14:50:42.323535 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:42.323542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:42.323627 (+ 85us) server_negotiation.cc:407] Connection header received
0928 14:50:42.323850 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:42.323854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:42.323937 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:42.324075 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:50:42.324498 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.325190 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.326506 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.326930 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.327039 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.327524 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:50:42.328032 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:50:42.328186 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:42.328238 (+ 52us) server_negotiation.cc:299] Negotiation successful
0928 14:50:42.328349 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:50:42.336544 19851 raft_consensus.cc:1261] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8 [term 1 FOLLOWER]: Refusing update from remote peer 73f68f45aa974e06956895264e912fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:42.336542 19584 raft_consensus.cc:1261] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c [term 1 FOLLOWER]: Refusing update from remote peer 73f68f45aa974e06956895264e912fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:42.337500 19925 consensus_queue.cc:1035] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f6a97bd327a44b78335c8380e15893c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:42.337822 19921 consensus_queue.cc:1035] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "bed9c021e3244db0990714ae72dda0c8" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 37175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:42.350684 19929 mvcc.cc:205] Tried to move back new op lower bound from 7334326445401128960 to 7334326445165862912. Current Snapshot: MvccSnapshot[applied={T|T < 7334326445401128960}]
I20260928 14:50:42.363020 19440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:42.363320 19440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:42.367316 19440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68bc8c9807f24fbb9fe055ebf379d11f: Sending DeleteTablet for 3 replicas of tablet 9c255edff8f04423ab0f106c9a83541c
I20260928 14:50:42.368666 19698 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c255edff8f04423ab0f106c9a83541c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:42 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:46820
I20260928 14:50:42.368754 19564 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c255edff8f04423ab0f106c9a83541c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:42 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:34472
I20260928 14:50:42.368934 19831 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c255edff8f04423ab0f106c9a83541c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:42 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:44342
I20260928 14:50:42.370450 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 19502
I20260928 14:50:42.380074 19940 tablet_replica.cc:330] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: stopping tablet replica
I20260928 14:50:42.380079 19938 tablet_replica.cc:330] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: stopping tablet replica
I20260928 14:50:42.380652 19940 raft_consensus.cc:2229] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:42.380707 19938 raft_consensus.cc:2229] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:50:42.381287 19940 raft_consensus.cc:2258] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:42.381345 19938 raft_consensus.cc:2258] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:42.383530 19940 ts_tablet_manager.cc:1905] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:42.383546 19938 ts_tablet_manager.cc:1905] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:42.390468 19940 ts_tablet_manager.cc:1918] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:50:42.390777 19940 log.cc:1196] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/9c255edff8f04423ab0f106c9a83541c
I20260928 14:50:42.390872 19938 ts_tablet_manager.cc:1918] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:50:42.391048 19938 log.cc:1196] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/9c255edff8f04423ab0f106c9a83541c
I20260928 14:50:42.391541 19940 ts_tablet_manager.cc:1939] T 9c255edff8f04423ab0f106c9a83541c P bed9c021e3244db0990714ae72dda0c8: Deleting consensus metadata
I20260928 14:50:42.391602 19938 ts_tablet_manager.cc:1939] T 9c255edff8f04423ab0f106c9a83541c P 73f68f45aa974e06956895264e912fbf: Deleting consensus metadata
I20260928 14:50:42.393570 19425 catalog_manager.cc:4841] TS bed9c021e3244db0990714ae72dda0c8 (127.14.241.195:37175): tablet 9c255edff8f04423ab0f106c9a83541c (table test-table [id=775b7e599f8444fda42c4a895ae88f94]) successfully deleted
I20260928 14:50:42.393891 19425 catalog_manager.cc:4841] TS 73f68f45aa974e06956895264e912fbf (127.14.241.194:42103): tablet 9c255edff8f04423ab0f106c9a83541c (table test-table [id=775b7e599f8444fda42c4a895ae88f94]) successfully deleted
I20260928 14:50:42.396873 19939 tablet_replica.cc:330] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: stopping tablet replica
I20260928 14:50:42.397367 19939 raft_consensus.cc:2229] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:42.397920 19939 raft_consensus.cc:2258] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:42.399945 19939 ts_tablet_manager.cc:1905] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:42.405349 19939 ts_tablet_manager.cc:1918] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:50:42.405535 19939 log.cc:1196] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/9c255edff8f04423ab0f106c9a83541c
I20260928 14:50:42.405946 19939 ts_tablet_manager.cc:1939] T 9c255edff8f04423ab0f106c9a83541c P 0f6a97bd327a44b78335c8380e15893c: Deleting consensus metadata
Sep 28 14:50:42 dist-test-slave-vk40 krb5kdc[19396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:42.407990 19427 catalog_manager.cc:4841] TS 0f6a97bd327a44b78335c8380e15893c (127.14.241.193:44081): tablet 9c255edff8f04423ab0f106c9a83541c (table test-table [id=775b7e599f8444fda42c4a895ae88f94]) successfully deleted
I20260928 14:50:42.413754 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.371804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34480 (local address 127.14.241.193:44081)
0928 14:50:42.371988 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:42.371994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:42.395993 (+ 23999us) server_negotiation.cc:407] Connection header received
0928 14:50:42.396188 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:42.396192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:42.396264 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:42.396431 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:42.399145 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.400270 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.402147 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.402749 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.402879 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.407911 (+ 5032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:42.407924 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:42.407938 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:42.407973 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:42.410704 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:42.411405 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:42.411412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:42.411423 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:42.411565 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:42.411945 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:42.411950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:42.411954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:42.412507 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:42.412798 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:42.413107 (+ 309us) server_negotiation.cc:299] Negotiation successful
0928 14:50:42.413325 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:50:42.414400 19625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:42.503749 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 19502
I20260928 14:50:42.518509 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 19636
Sep 28 14:50:42 dist-test-slave-vk40 krb5kdc[19396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:50:42.533994 19911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.519299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.14.241.194:42103)
0928 14:50:42.519493 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:42.519499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:42.519521 (+ 22us) server_negotiation.cc:407] Connection header received
0928 14:50:42.519586 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:42.519592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:42.519682 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:42.519787 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:42.520963 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.522177 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.523381 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.523988 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.524114 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.528090 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:42.528106 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:42.528121 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:42.528166 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:42.531094 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:42.531875 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:42.531883 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:42.531885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:42.532053 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:42.532486 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:42.532491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:42.532493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:42.532883 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:42.533140 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:42.533439 (+ 299us) server_negotiation.cc:299] Negotiation successful
0928 14:50:42.533661 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:50:42.534714 19759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:42.646929 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 19636
I20260928 14:50:42.664032 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 19769
Sep 28 14:50:42 dist-test-slave-vk40 krb5kdc[19396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:50:42.678092 19913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:42.664732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.14.241.195:37175)
0928 14:50:42.664934 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:42.664939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:42.664958 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:50:42.665070 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:42.665074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:42.665145 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:42.665224 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:42.666393 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.667118 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.668478 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:42.668870 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:42.668973 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:42.673035 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:42.673050 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:42.673065 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:42.673103 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:42.675708 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:42.676246 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:42.676253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:42.676255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:42.676531 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:42.676824 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:42.676827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:42.676829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:42.677194 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:42.677405 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:42.677671 (+ 266us) server_negotiation.cc:299] Negotiation successful
0928 14:50:42.677837 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 14:50:42.678710 19891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:42.766191 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 19769
I20260928 14:50:42.779103 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 19409
I20260928 14:50:42.780186 19471 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:42.888625 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 19409
2026-09-28T14:50:42Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4094 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-28T14:50:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:42Z Disabled control of system clock
I20260928 14:50:42.949008 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46309
--webserver_interface=127.14.241.254
--webserver_port=0
--builtin_ntp_servers=127.14.241.212:38473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:46309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 14:50:43.267197 19949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:43.267501 19949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:43.267584 19949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:43.274888 19949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:43.275003 19949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:43.275048 19949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:43.275102 19949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:43.275144 19949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:43.284495 19949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:38473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:46309
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:43.285755 19949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:43.287519 19949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:43.299014 19954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:43.298946 19957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:43.298967 19955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:43.299692 19949 server_base.cc:1034] running on GCE node
I20260928 14:50:43.300566 19949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:43.301963 19949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:43.303200 19949 hybrid_clock.cc:648] HybridClock initialized: now 1790607043303147 us; error 56 us; skew 500 ppm
I20260928 14:50:43.303555 19949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:50:43.305903 19949 webserver.cc:460] Webserver started at http://127.14.241.254:38259/ using document root <none> and password file <none>
I20260928 14:50:43.306622 19949 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:43.306735 19949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:43.307042 19949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:43.310001 19949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "e63c96ad3c744cbba5e0a2c872cbe38f"
format_stamp: "Formatted at 2026-09-28 14:50:43 on dist-test-slave-vk40"
server_key: "abbd7e2f91efe9a63fbd076c78780f3d"
server_key_iv: "d95134ed419955f23afe08b87c3aafdf"
server_key_version: "encryptionkey@0"
I20260928 14:50:43.310786 19949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "e63c96ad3c744cbba5e0a2c872cbe38f"
format_stamp: "Formatted at 2026-09-28 14:50:43 on dist-test-slave-vk40"
server_key: "abbd7e2f91efe9a63fbd076c78780f3d"
server_key_iv: "d95134ed419955f23afe08b87c3aafdf"
server_key_version: "encryptionkey@0"
I20260928 14:50:43.316102 19949 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 14:50:43.319810 19963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:43.321348 19949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:43.321537 19949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "e63c96ad3c744cbba5e0a2c872cbe38f"
format_stamp: "Formatted at 2026-09-28 14:50:43 on dist-test-slave-vk40"
server_key: "abbd7e2f91efe9a63fbd076c78780f3d"
server_key_iv: "d95134ed419955f23afe08b87c3aafdf"
server_key_version: "encryptionkey@0"
I20260928 14:50:43.321774 19949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:43.327623 19949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:43.328449 19949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:43.328719 19949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:43.345439 19949 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:46309
I20260928 14:50:43.345474 20014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:46309 every 8 connection(s)
I20260928 14:50:43.346776 19949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:43.350725 20015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:43.355269 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 19949
I20260928 14:50:43.355441 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:43.356050 15303 external_mini_cluster.cc:1428] Setting key 81975405bbc5c38c15972d4652522517
I20260928 14:50:43.362673 20015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f: Bootstrap starting.
I20260928 14:50:43.364709 20018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:43.358159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40992 (local address 127.14.241.254:46309)
0928 14:50:43.359207 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:43.359219 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:43.359259 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:50:43.361119 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:43.361144 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:43.361580 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:43.361965 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:43.362978 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:43.362985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:43.362998 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:43.363014 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:43.363072 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:43.363088 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:43.363298 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:43.363442 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 14:50:43.363695 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":335,"threads_started":1}
I20260928 14:50:43.366046 20015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:43.367111 20015 log.cc:824] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:43.369869 20015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f: No bootstrap required, opened a new log
I20260928 14:50:43.376965 20015 raft_consensus.cc:348] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63c96ad3c744cbba5e0a2c872cbe38f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46309 } }
I20260928 14:50:43.377357 20015 raft_consensus.cc:374] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:43.377447 20015 raft_consensus.cc:729] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63c96ad3c744cbba5e0a2c872cbe38f, State: Initialized, Role: FOLLOWER
I20260928 14:50:43.378007 20015 consensus_queue.cc:260] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63c96ad3c744cbba5e0a2c872cbe38f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46309 } }
I20260928 14:50:43.378222 20015 raft_consensus.cc:388] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:43.378345 20015 raft_consensus.cc:482] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:43.378489 20015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:43.380795 20015 raft_consensus.cc:504] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63c96ad3c744cbba5e0a2c872cbe38f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46309 } }
I20260928 14:50:43.381278 20015 leader_election.cc:302] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e63c96ad3c744cbba5e0a2c872cbe38f; no voters:
I20260928 14:50:43.381836 20015 leader_election.cc:288] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:43.381963 20020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:43.382601 20020 raft_consensus.cc:686] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [term 1 LEADER]: Becoming Leader. State: Replica: e63c96ad3c744cbba5e0a2c872cbe38f, State: Running, Role: LEADER
I20260928 14:50:43.383167 20020 consensus_queue.cc:237] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63c96ad3c744cbba5e0a2c872cbe38f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46309 } }
I20260928 14:50:43.384029 20015 sys_catalog.cc:564] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:43.385895 20021 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e63c96ad3c744cbba5e0a2c872cbe38f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63c96ad3c744cbba5e0a2c872cbe38f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46309 } } }
I20260928 14:50:43.385869 20022 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e63c96ad3c744cbba5e0a2c872cbe38f. Latest consensus state: current_term: 1 leader_uuid: "e63c96ad3c744cbba5e0a2c872cbe38f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63c96ad3c744cbba5e0a2c872cbe38f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46309 } } }
I20260928 14:50:43.386292 20021 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:43.386289 20022 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:43.388485 20027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:43.393567 20027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:43.401783 20027 catalog_manager.cc:1347] Generated new cluster ID: a90defe5b01348a39e2c41c68058ad56
I20260928 14:50:43.401906 20027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:43.427541 20027 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:43.428021 20027 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:43.436021 20027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f: Generated new TSK 0
I20260928 14:50:43.436935 20027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:50:43.451825 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46309
--builtin_ntp_servers=127.14.241.212:38473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 14:50:43.769793 20040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:43.770085 20040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:43.770217 20040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:43.777432 20040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:43.777549 20040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:43.777711 20040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:43.787065 20040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:38473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:43.788476 20040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:43.790298 20040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:43.803169 20048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:43.803216 20046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:43.803140 20045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:43.803941 20040 server_base.cc:1034] running on GCE node
I20260928 14:50:43.804638 20040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:43.805629 20040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:43.806838 20040 hybrid_clock.cc:648] HybridClock initialized: now 1790607043806792 us; error 59 us; skew 500 ppm
I20260928 14:50:43.807166 20040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:50:43.809463 20040 webserver.cc:460] Webserver started at http://127.14.241.193:34261/ using document root <none> and password file <none>
I20260928 14:50:43.810252 20040 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:43.810407 20040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:43.810722 20040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:43.813696 20040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "27a85e0142004901b9d14192fe07b0ce"
format_stamp: "Formatted at 2026-09-28 14:50:43 on dist-test-slave-vk40"
server_key: "9654ce9ba4375d9039a583883eb6811c"
server_key_iv: "27b4a901bf43263f9612866c5b03c3b1"
server_key_version: "encryptionkey@0"
I20260928 14:50:43.814481 20040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "27a85e0142004901b9d14192fe07b0ce"
format_stamp: "Formatted at 2026-09-28 14:50:43 on dist-test-slave-vk40"
server_key: "9654ce9ba4375d9039a583883eb6811c"
server_key_iv: "27b4a901bf43263f9612866c5b03c3b1"
server_key_version: "encryptionkey@0"
I20260928 14:50:43.819820 20040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260928 14:50:43.823324 20054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:43.824754 20040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:43.824944 20040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "27a85e0142004901b9d14192fe07b0ce"
format_stamp: "Formatted at 2026-09-28 14:50:43 on dist-test-slave-vk40"
server_key: "9654ce9ba4375d9039a583883eb6811c"
server_key_iv: "27b4a901bf43263f9612866c5b03c3b1"
server_key_version: "encryptionkey@0"
I20260928 14:50:43.825181 20040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:43.830713 20040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:43.831521 20040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:43.831781 20040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:43.832748 20040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:43.834095 20040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:43.834189 20040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:43.834295 20040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:43.834362 20040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:43.862454 20040 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:46395
I20260928 14:50:43.862500 20166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:46395 every 8 connection(s)
I20260928 14:50:43.863765 20040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:43.871615 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:43.866320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:49353 (local address 127.14.241.254:46309)
0928 14:50:43.866782 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:43.866792 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:43.868014 (+ 1222us) server_negotiation.cc:407] Connection header received
0928 14:50:43.869689 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:43.869693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:43.869746 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:43.869836 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:43.871019 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:43.871023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:43.871026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:43.871041 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:43.871081 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:43.871092 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:43.871179 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:43.871351 (+ 172us) server_negotiation.cc:299] Negotiation successful
0928 14:50:43.871402 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":206,"threads_started":1}
I20260928 14:50:43.871711 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 20040
I20260928 14:50:43.871912 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:50:43.872731 15303 external_mini_cluster.cc:1428] Setting key bc7ee4b18e1d77ba138fa9a2149cab36
I20260928 14:50:43.872980 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:43.866977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46309 (local address 127.14.241.193:49353)
0928 14:50:43.867853 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:43.867889 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:43.869435 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:43.870043 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:43.870052 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:43.870145 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:43.870724 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:50:43.870727 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:43.870772 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:50:43.870858 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:43.871162 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:43.871168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:43.871173 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:50:43.871447 (+ 274us) client_negotiation.cc:240] Negotiation successful
0928 14:50:43.871723 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":254,"threads_started":1}
I20260928 14:50:43.874532 20167 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46309
I20260928 14:50:43.874814 20167 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:43.875349 20167 heartbeater.cc:511] Master 127.14.241.254:46309 requested a full tablet report, sending...
I20260928 14:50:43.875427 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46309
--builtin_ntp_servers=127.14.241.212:38473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260928 14:50:43.877292 19980 ts_manager.cc:194] Registered new tserver with Master: 27a85e0142004901b9d14192fe07b0ce (127.14.241.193:46395)
W20260928 14:50:44.187777 20171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:44.188071 20171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:44.188184 20171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:44.195883 20171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:44.195998 20171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:44.196110 20171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:44.205525 20171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:38473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:44.206839 20171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:44.208582 20171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:44.221369 20178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:44.221390 20176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:44.221940 20180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:44.222405 20171 server_base.cc:1034] running on GCE node
I20260928 14:50:44.223029 20171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:44.223971 20171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:44.225164 20171 hybrid_clock.cc:648] HybridClock initialized: now 1790607044225118 us; error 39 us; skew 500 ppm
I20260928 14:50:44.225477 20171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:50:44.227694 20171 webserver.cc:460] Webserver started at http://127.14.241.194:42739/ using document root <none> and password file <none>
I20260928 14:50:44.228451 20171 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:44.228550 20171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:44.228859 20171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:44.231664 20171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "98ae3b2a87704458960bcab6ab3b13c8"
format_stamp: "Formatted at 2026-09-28 14:50:44 on dist-test-slave-vk40"
server_key: "9abe6a085b84fd548b99b57c35309c8e"
server_key_iv: "23a6d6b0caf2cf0914e3f426b96f7712"
server_key_version: "encryptionkey@0"
I20260928 14:50:44.232465 20171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "98ae3b2a87704458960bcab6ab3b13c8"
format_stamp: "Formatted at 2026-09-28 14:50:44 on dist-test-slave-vk40"
server_key: "9abe6a085b84fd548b99b57c35309c8e"
server_key_iv: "23a6d6b0caf2cf0914e3f426b96f7712"
server_key_version: "encryptionkey@0"
I20260928 14:50:44.237857 20171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:50:44.241309 20186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:44.242681 20171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 14:50:44.242839 20171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "98ae3b2a87704458960bcab6ab3b13c8"
format_stamp: "Formatted at 2026-09-28 14:50:44 on dist-test-slave-vk40"
server_key: "9abe6a085b84fd548b99b57c35309c8e"
server_key_iv: "23a6d6b0caf2cf0914e3f426b96f7712"
server_key_version: "encryptionkey@0"
I20260928 14:50:44.243057 20171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:44.248682 20171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:44.249477 20171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:44.249727 20171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:44.250607 20171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:44.252014 20171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:44.252112 20171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:44.252215 20171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:44.252302 20171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:44.281656 20171 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:42119
I20260928 14:50:44.281709 20298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:42119 every 8 connection(s)
I20260928 14:50:44.282977 20171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:44.291066 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:44.285614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:38395 (local address 127.14.241.254:46309)
0928 14:50:44.285843 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:44.285848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:44.287261 (+ 1413us) server_negotiation.cc:407] Connection header received
0928 14:50:44.288994 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:44.288999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:44.289055 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:44.289163 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:44.290456 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:44.290460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:44.290463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:44.290478 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:44.290505 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:44.290515 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:44.290609 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:44.290787 (+ 178us) server_negotiation.cc:299] Negotiation successful
0928 14:50:44.290837 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260928 14:50:44.292052 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:44.286166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46309 (local address 127.14.241.194:38395)
0928 14:50:44.287108 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:44.287145 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:44.288734 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:44.289380 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:44.289391 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:44.289505 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:44.290181 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:50:44.290184 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:44.290225 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:50:44.290316 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:44.290587 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:44.290592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:44.290598 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:50:44.290856 (+ 258us) client_negotiation.cc:240] Negotiation successful
0928 14:50:44.291066 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":263,"threads_started":1}
I20260928 14:50:44.292696 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 20171
I20260928 14:50:44.292842 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:50:44.293387 15303 external_mini_cluster.cc:1428] Setting key b094402271aed77ea1b39f561f1ab6a4
I20260928 14:50:44.293635 20299 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46309
I20260928 14:50:44.293927 20299 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:44.294580 20299 heartbeater.cc:511] Master 127.14.241.254:46309 requested a full tablet report, sending...
I20260928 14:50:44.295926 19980 ts_manager.cc:194] Registered new tserver with Master: 98ae3b2a87704458960bcab6ab3b13c8 (127.14.241.194:42119)
I20260928 14:50:44.296062 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46309
--builtin_ntp_servers=127.14.241.212:38473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 14:50:44.620486 20302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:44.620764 20302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:44.620852 20302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:44.628360 20302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:44.628466 20302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:44.628583 20302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:44.637987 20302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:38473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:44.639236 20302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:44.641032 20302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:44.653519 20307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:44.653556 20308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:44.653580 20310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:44.654377 20302 server_base.cc:1034] running on GCE node
I20260928 14:50:44.655011 20302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:44.655992 20302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:44.657184 20302 hybrid_clock.cc:648] HybridClock initialized: now 1790607044657126 us; error 50 us; skew 500 ppm
I20260928 14:50:44.657516 20302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:50:44.659680 20302 webserver.cc:460] Webserver started at http://127.14.241.195:41069/ using document root <none> and password file <none>
I20260928 14:50:44.660422 20302 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:44.660527 20302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:44.660840 20302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:44.663694 20302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "fcecd72c19534b549f140186b34fa757"
format_stamp: "Formatted at 2026-09-28 14:50:44 on dist-test-slave-vk40"
server_key: "685875133b004415fb1a1c2cd3ba03eb"
server_key_iv: "52442011a9d04cf968c549227ccd986d"
server_key_version: "encryptionkey@0"
I20260928 14:50:44.664553 20302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "fcecd72c19534b549f140186b34fa757"
format_stamp: "Formatted at 2026-09-28 14:50:44 on dist-test-slave-vk40"
server_key: "685875133b004415fb1a1c2cd3ba03eb"
server_key_iv: "52442011a9d04cf968c549227ccd986d"
server_key_version: "encryptionkey@0"
I20260928 14:50:44.669837 20302 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:50:44.673218 20316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:44.674503 20302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 14:50:44.674649 20302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "fcecd72c19534b549f140186b34fa757"
format_stamp: "Formatted at 2026-09-28 14:50:44 on dist-test-slave-vk40"
server_key: "685875133b004415fb1a1c2cd3ba03eb"
server_key_iv: "52442011a9d04cf968c549227ccd986d"
server_key_version: "encryptionkey@0"
I20260928 14:50:44.674852 20302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:44.680223 20302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:44.681053 20302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:44.681289 20302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:44.682180 20302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:44.683537 20302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:44.683627 20302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:44.683734 20302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:44.683833 20302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260928 14:50:44.713186 20302 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:46427
I20260928 14:50:44.713235 20429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:46427 every 8 connection(s)
I20260928 14:50:44.714421 20302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:44.722415 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:44.716928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:37267 (local address 127.14.241.254:46309)
0928 14:50:44.717145 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:44.717149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:44.718533 (+ 1384us) server_negotiation.cc:407] Connection header received
0928 14:50:44.720231 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:44.720237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:44.720330 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:44.720420 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:44.721704 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:44.721710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:44.721715 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:44.721734 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:44.721767 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:44.721779 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:44.721895 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:44.722066 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 14:50:44.722133 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 14:50:44.723344 20432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:44.717390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46309 (local address 127.14.241.195:37267)
0928 14:50:44.718366 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:44.718409 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:44.719936 (+ 1527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:44.720692 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:44.720700 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:44.720803 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:44.721398 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:50:44.721400 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:44.721438 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:50:44.721525 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:44.721867 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:44.721879 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:44.721886 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 14:50:44.722141 (+ 255us) client_negotiation.cc:240] Negotiation successful
0928 14:50:44.722424 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":308,"threads_started":1}
I20260928 14:50:44.724216 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 20302
I20260928 14:50:44.724390 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:50:44.724721 20430 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46309
I20260928 14:50:44.724817 15303 external_mini_cluster.cc:1428] Setting key 42725f39112a6e3fd1303606f99029c1
I20260928 14:50:44.725005 20430 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:44.725673 20430 heartbeater.cc:511] Master 127.14.241.254:46309 requested a full tablet report, sending...
I20260928 14:50:44.727043 19979 ts_manager.cc:194] Registered new tserver with Master: fcecd72c19534b549f140186b34fa757 (127.14.241.195:46427)
I20260928 14:50:44.739723 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:50:44.746399 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:44.744116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.14.241.254:46309)
0928 14:50:44.744388 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:44.744396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:44.744776 (+ 380us) server_negotiation.cc:407] Connection header received
0928 14:50:44.744922 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:44.744926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:44.744983 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:44.745059 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:44.745840 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:44.745844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:44.745847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:44.745862 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:44.745895 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:44.745906 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:44.746021 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:44.746066 (+ 45us) server_negotiation.cc:299] Negotiation successful
0928 14:50:44.746115 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260928 14:50:44.750337 19979 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:50:44.756906 19979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:44.772665 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:44.768447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:46427 (local address 127.0.0.1:44546)
0928 14:50:44.769082 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:44.769129 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:44.769253 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:44.770495 (+ 1242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:44.770503 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:44.770605 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:44.771406 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:50:44.771409 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:44.771454 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:50:44.771531 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:44.771964 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:44.771975 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:44.771982 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 14:50:44.772134 (+ 152us) client_negotiation.cc:240] Negotiation successful
0928 14:50:44.772217 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":618,"threads_started":1}
I20260928 14:50:44.773519 20443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:44.768811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44546 (local address 127.14.241.195:46427)
0928 14:50:44.769595 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:44.769608 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:44.769634 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:50:44.769700 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:44.769708 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:44.769972 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:44.770132 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:44.771708 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:44.771717 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:44.771733 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:44.771751 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:44.771820 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:44.771842 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:44.772953 (+ 1111us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:44.773024 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 14:50:44.773123 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":1013,"threads_started":1}
I20260928 14:50:44.774612 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:44.769727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:42119 (local address 127.0.0.1:36656)
0928 14:50:44.770237 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:44.770254 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:44.770337 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:44.772428 (+ 2091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:44.772432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:44.772459 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:44.773156 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:50:44.773159 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:44.773176 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:50:44.773220 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:44.774066 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:44.774071 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:44.774076 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:50:44.774211 (+ 135us) client_negotiation.cc:240] Negotiation successful
0928 14:50:44.774261 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":278,"threads_started":1}
I20260928 14:50:44.774680 20445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:44.770102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36656 (local address 127.14.241.194:42119)
0928 14:50:44.771879 (+ 1777us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:44.771887 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:44.771926 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:50:44.771991 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:44.771999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:44.772215 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:44.772875 (+ 660us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:44.773877 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:44.773883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:44.773892 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:44.773908 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:44.773954 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:44.773968 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:44.774088 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:44.774235 (+ 147us) server_negotiation.cc:299] Negotiation successful
0928 14:50:44.774332 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1617,"thread_start_us":2307,"threads_started":1}
I20260928 14:50:44.776613 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:44.770312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:46395 (local address 127.0.0.1:53238)
0928 14:50:44.771340 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:44.771355 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:44.771444 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:44.774943 (+ 3499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:44.774947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:44.774968 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:44.775578 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:50:44.775582 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:44.775598 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:50:44.775640 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:44.776044 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:44.776049 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:44.776054 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:50:44.776153 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 14:50:44.776200 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"thread_start_us":996,"threads_started":1}
I20260928 14:50:44.777199 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:44.770843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53238 (local address 127.14.241.193:46395)
0928 14:50:44.774458 (+ 3615us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:44.774467 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:44.774493 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:50:44.774561 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:44.774569 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:44.774800 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:44.774959 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:44.775807 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:44.775814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:44.775827 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:44.775846 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:44.775902 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:44.775927 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:44.776862 (+ 935us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:44.776910 (+ 48us) server_negotiation.cc:299] Negotiation successful
0928 14:50:44.776994 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3462,"thread_start_us":3584,"threads_started":1}
I20260928 14:50:44.777786 20364 tablet_service.cc:1467] Processing CreateTablet for tablet 934e161f765a4a11bdd38eb5470a5af9 (DEFAULT_TABLE table=test-table [id=4777ce7529804c2989b245d6dbc10c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:44.779320 20364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 934e161f765a4a11bdd38eb5470a5af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:44.779414 20234 tablet_service.cc:1467] Processing CreateTablet for tablet 934e161f765a4a11bdd38eb5470a5af9 (DEFAULT_TABLE table=test-table [id=4777ce7529804c2989b245d6dbc10c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:44.780143 20102 tablet_service.cc:1467] Processing CreateTablet for tablet 934e161f765a4a11bdd38eb5470a5af9 (DEFAULT_TABLE table=test-table [id=4777ce7529804c2989b245d6dbc10c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:44.780769 20234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 934e161f765a4a11bdd38eb5470a5af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:44.781219 20102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 934e161f765a4a11bdd38eb5470a5af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:44.792071 20448 tablet_bootstrap.cc:492] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: Bootstrap starting.
I20260928 14:50:44.794142 20449 tablet_bootstrap.cc:492] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: Bootstrap starting.
I20260928 14:50:44.795677 20450 tablet_bootstrap.cc:492] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: Bootstrap starting.
I20260928 14:50:44.796530 20448 tablet_bootstrap.cc:654] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:44.797688 20449 tablet_bootstrap.cc:654] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:44.797854 20448 log.cc:824] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:44.798734 20450 tablet_bootstrap.cc:654] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:44.798892 20449 log.cc:824] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:44.799934 20450 log.cc:824] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:44.801060 20448 tablet_bootstrap.cc:492] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: No bootstrap required, opened a new log
I20260928 14:50:44.801347 20448 ts_tablet_manager.cc:1397] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260928 14:50:44.801885 20449 tablet_bootstrap.cc:492] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: No bootstrap required, opened a new log
I20260928 14:50:44.802150 20449 ts_tablet_manager.cc:1397] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260928 14:50:44.802188 20450 tablet_bootstrap.cc:492] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: No bootstrap required, opened a new log
I20260928 14:50:44.802400 20450 ts_tablet_manager.cc:1397] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20260928 14:50:44.809335 20450 raft_consensus.cc:348] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcecd72c19534b549f140186b34fa757" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46427 } } peers { permanent_uuid: "98ae3b2a87704458960bcab6ab3b13c8" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42119 } } peers { permanent_uuid: "27a85e0142004901b9d14192fe07b0ce" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 46395 } }
I20260928 14:50:44.809705 20450 raft_consensus.cc:374] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:44.809813 20450 raft_consensus.cc:729] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ae3b2a87704458960bcab6ab3b13c8, State: Initialized, Role: FOLLOWER
I20260928 14:50:44.810395 20450 consensus_queue.cc:260] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcecd72c19534b549f140186b34fa757" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46427 } } peers { permanent_uuid: "98ae3b2a87704458960bcab6ab3b13c8" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42119 } } peers { permanent_uuid: "27a85e0142004901b9d14192fe07b0ce" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 46395 } }
I20260928 14:50:44.811416 20299 heartbeater.cc:503] Master 127.14.241.254:46309 was elected leader, sending a full tablet report...
I20260928 14:50:44.811635 20448 raft_consensus.cc:348] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcecd72c19534b549f140186b34fa757" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46427 } } peers { permanent_uuid: "98ae3b2a87704458960bcab6ab3b13c8" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42119 } } peers { permanent_uuid: "27a85e0142004901b9d14192fe07b0ce" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 46395 } }
I20260928 14:50:44.812070 20448 raft_consensus.cc:374] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:44.812098 20450 ts_tablet_manager.cc:1428] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: Time spent starting tablet: real 0.010s user 0.008s sys 0.002s
I20260928 14:50:44.812152 20448 raft_consensus.cc:729] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a85e0142004901b9d14192fe07b0ce, State: Initialized, Role: FOLLOWER
I20260928 14:50:44.812052 20449 raft_consensus.cc:348] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcecd72c19534b549f140186b34fa757" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46427 } } peers { permanent_uuid: "98ae3b2a87704458960bcab6ab3b13c8" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42119 } } peers { permanent_uuid: "27a85e0142004901b9d14192fe07b0ce" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 46395 } }
I20260928 14:50:44.812507 20449 raft_consensus.cc:374] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:44.812647 20449 raft_consensus.cc:729] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcecd72c19534b549f140186b34fa757, State: Initialized, Role: FOLLOWER
I20260928 14:50:44.812763 20448 consensus_queue.cc:260] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcecd72c19534b549f140186b34fa757" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46427 } } peers { permanent_uuid: "98ae3b2a87704458960bcab6ab3b13c8" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42119 } } peers { permanent_uuid: "27a85e0142004901b9d14192fe07b0ce" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 46395 } }
I20260928 14:50:44.813360 20449 consensus_queue.cc:260] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcecd72c19534b549f140186b34fa757" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46427 } } peers { permanent_uuid: "98ae3b2a87704458960bcab6ab3b13c8" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42119 } } peers { permanent_uuid: "27a85e0142004901b9d14192fe07b0ce" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 46395 } }
I20260928 14:50:44.814218 20167 heartbeater.cc:503] Master 127.14.241.254:46309 was elected leader, sending a full tablet report...
I20260928 14:50:44.814918 20448 ts_tablet_manager.cc:1428] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260928 14:50:44.816040 20430 heartbeater.cc:503] Master 127.14.241.254:46309 was elected leader, sending a full tablet report...
I20260928 14:50:44.816668 20449 ts_tablet_manager.cc:1428] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
W20260928 14:50:44.865918 20168 tablet.cc:2369] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:50:44.965961 20431 tablet.cc:2369] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:50:45.034945 20300 tablet.cc:2369] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:50:45.214617 20456 raft_consensus.cc:482] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:50:45.214933 20456 raft_consensus.cc:504] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcecd72c19534b549f140186b34fa757" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46427 } } peers { permanent_uuid: "98ae3b2a87704458960bcab6ab3b13c8" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42119 } } peers { permanent_uuid: "27a85e0142004901b9d14192fe07b0ce" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 46395 } }
I20260928 14:50:45.216543 20456 leader_election.cc:288] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98ae3b2a87704458960bcab6ab3b13c8 (127.14.241.194:42119), 27a85e0142004901b9d14192fe07b0ce (127.14.241.193:46395)
I20260928 14:50:45.219182 20432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:45.217003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:42119 (local address 127.14.241.195:34847)
0928 14:50:45.217182 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:45.217202 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:45.217323 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:45.217745 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:45.217750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:45.217775 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:45.218397 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:50:45.218401 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:45.218421 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:50:45.218472 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:45.218806 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:45.218811 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:45.218816 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:50:45.218913 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 14:50:45.218956 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 14:50:45.219213 20445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:45.217124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:34847 (local address 127.14.241.194:42119)
0928 14:50:45.217302 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:45.217307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:45.217330 (+ 23us) server_negotiation.cc:407] Connection header received
0928 14:50:45.217499 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:45.217504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:45.217568 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:45.217667 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:45.218649 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:45.218655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:45.218658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:45.218674 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:45.218705 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:45.218716 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:45.218805 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:45.218925 (+ 120us) server_negotiation.cc:299] Negotiation successful
0928 14:50:45.218981 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 14:50:45.220085 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:45.217590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:59799 (local address 127.14.241.193:46395)
0928 14:50:45.217751 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:45.217756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:45.218218 (+ 462us) server_negotiation.cc:407] Connection header received
0928 14:50:45.218322 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:45.218328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:45.218410 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:45.218500 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:45.219436 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:45.219441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:45.219446 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:45.219465 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:45.219501 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:45.219515 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:45.219581 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:45.219756 (+ 175us) server_negotiation.cc:299] Negotiation successful
0928 14:50:45.219814 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 14:50:45.220211 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:45.217411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:46395 (local address 127.14.241.195:59799)
0928 14:50:45.218115 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:45.218129 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:45.218222 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:45.218557 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:45.218562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:45.218585 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:45.219202 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:50:45.219205 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:45.219223 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:50:45.219266 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:45.219644 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:45.219647 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:45.219649 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:50:45.219752 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 14:50:45.219803 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":447,"threads_started":1}
I20260928 14:50:45.220091 20254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "934e161f765a4a11bdd38eb5470a5af9" candidate_uuid: "fcecd72c19534b549f140186b34fa757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ae3b2a87704458960bcab6ab3b13c8" is_pre_election: true
I20260928 14:50:45.220677 20254 raft_consensus.cc:2454] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcecd72c19534b549f140186b34fa757 in term 0.
I20260928 14:50:45.221179 20122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "934e161f765a4a11bdd38eb5470a5af9" candidate_uuid: "fcecd72c19534b549f140186b34fa757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a85e0142004901b9d14192fe07b0ce" is_pre_election: true
I20260928 14:50:45.221382 20318 leader_election.cc:302] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ae3b2a87704458960bcab6ab3b13c8, fcecd72c19534b549f140186b34fa757; no voters:
I20260928 14:50:45.221657 20122 raft_consensus.cc:2454] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcecd72c19534b549f140186b34fa757 in term 0.
I20260928 14:50:45.221751 20456 raft_consensus.cc:2781] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:50:45.221879 20456 raft_consensus.cc:482] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:50:45.221982 20456 raft_consensus.cc:3037] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:45.224097 20456 raft_consensus.cc:504] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcecd72c19534b549f140186b34fa757" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46427 } } peers { permanent_uuid: "98ae3b2a87704458960bcab6ab3b13c8" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42119 } } peers { permanent_uuid: "27a85e0142004901b9d14192fe07b0ce" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 46395 } }
I20260928 14:50:45.224689 20456 leader_election.cc:288] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [CANDIDATE]: Term 1 election: Requested vote from peers 98ae3b2a87704458960bcab6ab3b13c8 (127.14.241.194:42119), 27a85e0142004901b9d14192fe07b0ce (127.14.241.193:46395)
I20260928 14:50:45.225098 20254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "934e161f765a4a11bdd38eb5470a5af9" candidate_uuid: "fcecd72c19534b549f140186b34fa757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ae3b2a87704458960bcab6ab3b13c8"
I20260928 14:50:45.225193 20122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "934e161f765a4a11bdd38eb5470a5af9" candidate_uuid: "fcecd72c19534b549f140186b34fa757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a85e0142004901b9d14192fe07b0ce"
I20260928 14:50:45.225313 20254 raft_consensus.cc:3037] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:45.225383 20122 raft_consensus.cc:3037] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:45.227557 20254 raft_consensus.cc:2454] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcecd72c19534b549f140186b34fa757 in term 1.
I20260928 14:50:45.227573 20122 raft_consensus.cc:2454] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcecd72c19534b549f140186b34fa757 in term 1.
I20260928 14:50:45.227962 20318 leader_election.cc:302] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a85e0142004901b9d14192fe07b0ce, fcecd72c19534b549f140186b34fa757; no voters:
I20260928 14:50:45.228210 20456 raft_consensus.cc:2781] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:45.228724 20456 raft_consensus.cc:686] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 1 LEADER]: Becoming Leader. State: Replica: fcecd72c19534b549f140186b34fa757, State: Running, Role: LEADER
I20260928 14:50:45.229189 20456 consensus_queue.cc:237] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcecd72c19534b549f140186b34fa757" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46427 } } peers { permanent_uuid: "98ae3b2a87704458960bcab6ab3b13c8" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42119 } } peers { permanent_uuid: "27a85e0142004901b9d14192fe07b0ce" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 46395 } }
I20260928 14:50:45.233757 19980 catalog_manager.cc:5495] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcecd72c19534b549f140186b34fa757 (127.14.241.195). New cstate: current_term: 1 leader_uuid: "fcecd72c19534b549f140186b34fa757" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcecd72c19534b549f140186b34fa757" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 46427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ae3b2a87704458960bcab6ab3b13c8" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27a85e0142004901b9d14192fe07b0ce" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 46395 } health_report { overall_health: UNKNOWN } } }
I20260928 14:50:45.347728 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:45.344870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.14.241.195:46427)
0928 14:50:45.345349 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:45.345356 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:45.345734 (+ 378us) server_negotiation.cc:407] Connection header received
0928 14:50:45.345948 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:45.345953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:45.346034 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:45.346125 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:45.347069 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:45.347075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:45.347080 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:45.347098 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:45.347139 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:45.347151 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:45.347246 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:45.347341 (+ 95us) server_negotiation.cc:299] Negotiation successful
0928 14:50:45.347399 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":250,"threads_started":1}
I20260928 14:50:45.356559 20122 raft_consensus.cc:1261] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce [term 1 FOLLOWER]: Refusing update from remote peer fcecd72c19534b549f140186b34fa757: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:45.356662 20254 raft_consensus.cc:1261] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8 [term 1 FOLLOWER]: Refusing update from remote peer fcecd72c19534b549f140186b34fa757: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:50:45.357604 20458 consensus_queue.cc:1035] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ae3b2a87704458960bcab6ab3b13c8" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:45.357983 20456 consensus_queue.cc:1035] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27a85e0142004901b9d14192fe07b0ce" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 46395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:50:45.368942 20463 mvcc.cc:205] Tried to move back new op lower bound from 7334326457770258432 to 7334326457262399488. Current Snapshot: MvccSnapshot[applied={T|T < 7334326457770258432}]
I20260928 14:50:45.369644 20466 mvcc.cc:205] Tried to move back new op lower bound from 7334326457770258432 to 7334326457262399488. Current Snapshot: MvccSnapshot[applied={T|T < 7334326457770258432}]
I20260928 14:50:45.383949 19980 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:45.384187 19980 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:50:45.387867 19980 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e63c96ad3c744cbba5e0a2c872cbe38f: Sending DeleteTablet for 3 replicas of tablet 934e161f765a4a11bdd38eb5470a5af9
I20260928 14:50:45.389002 20364 tablet_service.cc:1514] Processing DeleteTablet for tablet 934e161f765a4a11bdd38eb5470a5af9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:45 UTC) from {username='slave'} at 127.0.0.1:44546
I20260928 14:50:45.389213 20102 tablet_service.cc:1514] Processing DeleteTablet for tablet 934e161f765a4a11bdd38eb5470a5af9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:45 UTC) from {username='slave'} at 127.0.0.1:53238
I20260928 14:50:45.389219 20234 tablet_service.cc:1514] Processing DeleteTablet for tablet 934e161f765a4a11bdd38eb5470a5af9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:50:45 UTC) from {username='slave'} at 127.0.0.1:36656
I20260928 14:50:45.391260 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 20040
I20260928 14:50:45.398173 20476 tablet_replica.cc:330] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: stopping tablet replica
I20260928 14:50:45.398710 20476 raft_consensus.cc:2229] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:45.399520 20476 raft_consensus.cc:2258] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:45.400928 20475 tablet_replica.cc:330] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: stopping tablet replica
I20260928 14:50:45.402685 20476 ts_tablet_manager.cc:1905] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:45.403334 20475 raft_consensus.cc:2229] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:50:45.404174 20475 raft_consensus.cc:2258] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:45.412593 20476 ts_tablet_manager.cc:1918] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:50:45.412909 20476 log.cc:1196] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/934e161f765a4a11bdd38eb5470a5af9
I20260928 14:50:45.413661 20476 ts_tablet_manager.cc:1939] T 934e161f765a4a11bdd38eb5470a5af9 P 27a85e0142004901b9d14192fe07b0ce: Deleting consensus metadata
I20260928 14:50:45.413797 20475 ts_tablet_manager.cc:1905] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:45.415365 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:45.392747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53246 (local address 127.14.241.193:46395)
0928 14:50:45.392926 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:45.392932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:45.406177 (+ 13245us) server_negotiation.cc:407] Connection header received
0928 14:50:45.406719 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:45.406726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:45.406845 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:45.406927 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:45.414482 (+ 7555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:45.414490 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:45.414495 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:45.414522 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:45.414564 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:45.414581 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:45.414687 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:45.414937 (+ 250us) server_negotiation.cc:299] Negotiation successful
0928 14:50:45.415007 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:50:45.415964 19965 catalog_manager.cc:4841] TS 27a85e0142004901b9d14192fe07b0ce (127.14.241.193:46395): tablet 934e161f765a4a11bdd38eb5470a5af9 (table test-table [id=4777ce7529804c2989b245d6dbc10c87]) successfully deleted
I20260928 14:50:45.416625 20162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:45.424829 20477 tablet_replica.cc:330] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: stopping tablet replica
I20260928 14:50:45.425401 20477 raft_consensus.cc:2229] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:45.426122 20477 raft_consensus.cc:2258] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:45.428537 20477 ts_tablet_manager.cc:1905] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:45.429975 20475 ts_tablet_manager.cc:1918] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:50:45.430248 20475 log.cc:1196] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/934e161f765a4a11bdd38eb5470a5af9
I20260928 14:50:45.430871 20475 ts_tablet_manager.cc:1939] T 934e161f765a4a11bdd38eb5470a5af9 P fcecd72c19534b549f140186b34fa757: Deleting consensus metadata
I20260928 14:50:45.432874 19965 catalog_manager.cc:4841] TS fcecd72c19534b549f140186b34fa757 (127.14.241.195:46427): tablet 934e161f765a4a11bdd38eb5470a5af9 (table test-table [id=4777ce7529804c2989b245d6dbc10c87]) successfully deleted
I20260928 14:50:45.435760 20477 ts_tablet_manager.cc:1918] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:50:45.435927 20477 log.cc:1196] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/934e161f765a4a11bdd38eb5470a5af9
I20260928 14:50:45.436481 20477 ts_tablet_manager.cc:1939] T 934e161f765a4a11bdd38eb5470a5af9 P 98ae3b2a87704458960bcab6ab3b13c8: Deleting consensus metadata
I20260928 14:50:45.438457 19965 catalog_manager.cc:4841] TS 98ae3b2a87704458960bcab6ab3b13c8 (127.14.241.194:42119): tablet 934e161f765a4a11bdd38eb5470a5af9 (table test-table [id=4777ce7529804c2989b245d6dbc10c87]) successfully deleted
I20260928 14:50:45.507711 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 20040
I20260928 14:50:45.521733 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 20171
I20260928 14:50:45.524641 20445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:45.522507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.14.241.194:42119)
0928 14:50:45.522683 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:45.522688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:45.522727 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:50:45.522962 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:45.522966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:45.523058 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:45.523147 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:45.524024 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:45.524028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:45.524032 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:45.524048 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:45.524080 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:45.524092 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:45.524207 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:45.524311 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 14:50:45.524376 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:50:45.525341 20294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:45.611419 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 20171
I20260928 14:50:45.625353 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 20302
I20260928 14:50:45.628266 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:45.626126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44578 (local address 127.14.241.195:46427)
0928 14:50:45.626370 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:45.626386 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:45.626435 (+ 49us) server_negotiation.cc:407] Connection header received
0928 14:50:45.626626 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:45.626632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:45.626714 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:45.626827 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:45.627659 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:45.627665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:45.627669 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:45.627690 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:45.627728 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:45.627748 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:45.627880 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:45.627926 (+ 46us) server_negotiation.cc:299] Negotiation successful
0928 14:50:45.627981 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260928 14:50:45.629097 20425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:45.745205 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 20302
I20260928 14:50:45.761091 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 19949
I20260928 14:50:45.762331 20010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:45.870095 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 19949
2026-09-28T14:50:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2980 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260928 14:50:45.907795 15303 mini_oidc.cc:152] Starting JWKS server
I20260928 14:50:45.910410 15303 webserver.cc:460] Webserver started at https://127.0.0.1:40563/ using document root <none> and password file <none>
I20260928 14:50:45.910728 15303 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 14:50:45.911401 15303 webserver.cc:460] Webserver started at http://127.0.0.1:44077/ using document root <none> and password file <none>
2026-09-28T14:50:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:45Z Disabled control of system clock
I20260928 14:50:45.936720 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:36449
--webserver_interface=127.14.241.254
--webserver_port=0
--builtin_ntp_servers=127.14.241.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:36449
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40563/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/testchainca.cert with env {}
W20260928 14:50:46.260855 20488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:46.261145 20488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:46.261245 20488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:46.268491 20488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:46.268604 20488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:46.268647 20488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:46.268710 20488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:46.268754 20488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:50:46.268817 20488 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:50:46.268855 20488 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40563/jwks.json
I20260928 14:50:46.279038 20488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:36449
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:36449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40563/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:46.280367 20488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:46.281657 20488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:46.293843 20496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:46.293861 20494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:46.294118 20493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:46.294561 20488 server_base.cc:1034] running on GCE node
I20260928 14:50:46.295336 20488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:46.296900 20488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:46.298098 20488 hybrid_clock.cc:648] HybridClock initialized: now 1790607046298051 us; error 44 us; skew 500 ppm
I20260928 14:50:46.298465 20488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:50:46.300868 20488 webserver.cc:460] Webserver started at http://127.14.241.254:40855/ using document root <none> and password file <none>
I20260928 14:50:46.301565 20488 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:46.301664 20488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:46.301971 20488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:46.304997 20488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "4387121909a64b9f95b73efbe27f92b5"
format_stamp: "Formatted at 2026-09-28 14:50:46 on dist-test-slave-vk40"
server_key: "1c616b6a1e5eedf55365b8ba7fc7dcad"
server_key_iv: "6533b3cbe62e5cac58cf1327704379a9"
server_key_version: "encryptionkey@0"
I20260928 14:50:46.305801 20488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "4387121909a64b9f95b73efbe27f92b5"
format_stamp: "Formatted at 2026-09-28 14:50:46 on dist-test-slave-vk40"
server_key: "1c616b6a1e5eedf55365b8ba7fc7dcad"
server_key_iv: "6533b3cbe62e5cac58cf1327704379a9"
server_key_version: "encryptionkey@0"
I20260928 14:50:46.311028 20488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:50:46.314697 20502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:46.316107 20488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:50:46.316272 20488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "4387121909a64b9f95b73efbe27f92b5"
format_stamp: "Formatted at 2026-09-28 14:50:46 on dist-test-slave-vk40"
server_key: "1c616b6a1e5eedf55365b8ba7fc7dcad"
server_key_iv: "6533b3cbe62e5cac58cf1327704379a9"
server_key_version: "encryptionkey@0"
I20260928 14:50:46.316521 20488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:46.330734 20488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:46.331588 20488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:46.331871 20488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:46.349411 20488 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:36449
I20260928 14:50:46.349432 20553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:36449 every 8 connection(s)
I20260928 14:50:46.350790 20488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:46.354538 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 20488
I20260928 14:50:46.354746 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:46.354838 20554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:46.355366 15303 external_mini_cluster.cc:1428] Setting key 364b41403474c7df794f929055edf687
I20260928 14:50:46.370548 20554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5: Bootstrap starting.
I20260928 14:50:46.371129 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:46.357235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56076 (local address 127.14.241.254:36449)
0928 14:50:46.358463 (+ 1228us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:46.358480 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:46.358533 (+ 53us) server_negotiation.cc:407] Connection header received
0928 14:50:46.360237 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:46.360268 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:46.361003 (+ 735us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:46.361505 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:46.362125 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:46.364141 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:46.365662 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:46.366765 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:46.366994 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:46.368075 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:46.368089 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:46.368116 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:46.368921 (+ 805us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:46.369039 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:46.369069 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:46.369182 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:46.369419 (+ 237us) server_negotiation.cc:299] Negotiation successful
0928 14:50:46.369801 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":378,"threads_started":1}
I20260928 14:50:46.374100 20554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:46.375150 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:50:46.375440 20554 log.cc:824] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:46.378809 20554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5: No bootstrap required, opened a new log
I20260928 14:50:46.386487 20554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387121909a64b9f95b73efbe27f92b5" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 36449 } }
I20260928 14:50:46.386911 20554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:46.386987 20554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4387121909a64b9f95b73efbe27f92b5, State: Initialized, Role: FOLLOWER
I20260928 14:50:46.387502 20554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387121909a64b9f95b73efbe27f92b5" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 36449 } }
I20260928 14:50:46.387734 20554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:46.387874 20554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:46.388055 20554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:46.390379 20554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387121909a64b9f95b73efbe27f92b5" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 36449 } }
I20260928 14:50:46.390899 20554 leader_election.cc:302] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4387121909a64b9f95b73efbe27f92b5; no voters:
I20260928 14:50:46.391441 20554 leader_election.cc:288] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:46.391592 20559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:46.392244 20559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [term 1 LEADER]: Becoming Leader. State: Replica: 4387121909a64b9f95b73efbe27f92b5, State: Running, Role: LEADER
I20260928 14:50:46.392777 20559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387121909a64b9f95b73efbe27f92b5" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 36449 } }
I20260928 14:50:46.393420 20554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:46.395200 20561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4387121909a64b9f95b73efbe27f92b5. Latest consensus state: current_term: 1 leader_uuid: "4387121909a64b9f95b73efbe27f92b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387121909a64b9f95b73efbe27f92b5" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 36449 } } }
I20260928 14:50:46.395635 20561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:46.395540 20560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4387121909a64b9f95b73efbe27f92b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387121909a64b9f95b73efbe27f92b5" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 36449 } } }
I20260928 14:50:46.395792 20560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:46.396534 20567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:46.401573 20567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:46.409775 20567 catalog_manager.cc:1347] Generated new cluster ID: 4143d32f5bcc4675bd3d98906da2e038
I20260928 14:50:46.409910 20567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:46.432658 20567 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:46.434162 20567 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:46.441071 20567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4387121909a64b9f95b73efbe27f92b5: Generated new TSK 0
I20260928 14:50:46.442278 20567 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 14:50:46.479709 20585 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260928 14:50:46.479871 20585 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 14:50:46.481141 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:46.459042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56086 (local address 127.14.241.254:36449)
0928 14:50:46.459245 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:46.459252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:46.459591 (+ 339us) server_negotiation.cc:407] Connection header received
0928 14:50:46.459891 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:46.459897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:46.459995 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:46.460157 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:50:46.460735 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:46.461919 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:46.463477 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:46.464148 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:46.464310 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:46.464933 (+ 623us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:50:46.480404 (+ 15471us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40563/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 14:50:46.480810 (+ 406us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56086: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40563/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":79}
W20260928 14:50:46.481276 20557 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56086: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40563/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260928 14:50:46.482585 20584 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:50:46.458846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:36449 (local address 127.0.0.1:56086)
0928 14:50:46.459483 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:46.459499 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:46.459655 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:46.460209 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:46.460213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:46.460223 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:50:46.460576 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:46.460586 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:46.462100 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:46.462104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:46.463316 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:46.463331 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:46.464396 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:46.464402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:46.464538 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:46.464606 (+ 68us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:50:46.480761 (+ 16155us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40563/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 14:50:46.481060 (+ 299us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.241.254:36449: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40563/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":330,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.14.241.254:36449: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40563/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260928 14:50:46.484443 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 20488
I20260928 14:50:46.485725 20549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:46.579944 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 20488
2026-09-28T14:50:46Z chronyd exiting
I20260928 14:50:46.627501 15303 test_util.cc:175] -----------------------------------------------
I20260928 14:50:46.627645 15303 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790607004819300-15303-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (724 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260928 14:50:46.631237 15303 mini_oidc.cc:152] Starting JWKS server
I20260928 14:50:46.633067 15303 webserver.cc:460] Webserver started at https://127.0.0.1:38847/ using document root <none> and password file <none>
I20260928 14:50:46.633265 15303 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 14:50:46.633895 15303 webserver.cc:460] Webserver started at http://127.0.0.1:38011/ using document root <none> and password file <none>
2026-09-28T14:50:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:46Z Disabled control of system clock
I20260928 14:50:46.659328 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41935
--webserver_interface=127.14.241.254
--webserver_port=0
--builtin_ntp_servers=127.14.241.212:44333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:41935
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38847/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/testchainca.cert with env {}
W20260928 14:50:46.978223 20592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:46.978533 20592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:46.978621 20592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:46.985988 20592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:46.986100 20592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:46.986145 20592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:46.986198 20592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:46.986243 20592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:50:46.986306 20592 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:50:46.986344 20592 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38847/jwks.json
I20260928 14:50:46.996511 20592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:41935
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38847/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:46.997817 20592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:46.999162 20592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:47.010656 20597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:47.010614 20600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:47.010692 20598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:47.011255 20592 server_base.cc:1034] running on GCE node
I20260928 14:50:47.012087 20592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:47.013612 20592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:47.014839 20592 hybrid_clock.cc:648] HybridClock initialized: now 1790607047014793 us; error 37 us; skew 500 ppm
I20260928 14:50:47.015249 20592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:50:47.017661 20592 webserver.cc:460] Webserver started at http://127.14.241.254:42713/ using document root <none> and password file <none>
I20260928 14:50:47.018374 20592 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:47.018479 20592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:47.018796 20592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:47.021816 20592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "2dc27c2a2d6e4efc865aa0bb506d8a37"
format_stamp: "Formatted at 2026-09-28 14:50:47 on dist-test-slave-vk40"
server_key: "22020c3d61df4f9a0d5f32f7b6cf8d48"
server_key_iv: "a0ff9b4eec1dfdf082010334fa13cf0a"
server_key_version: "encryptionkey@0"
I20260928 14:50:47.022641 20592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "2dc27c2a2d6e4efc865aa0bb506d8a37"
format_stamp: "Formatted at 2026-09-28 14:50:47 on dist-test-slave-vk40"
server_key: "22020c3d61df4f9a0d5f32f7b6cf8d48"
server_key_iv: "a0ff9b4eec1dfdf082010334fa13cf0a"
server_key_version: "encryptionkey@0"
I20260928 14:50:47.028080 20592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 14:50:47.031788 20606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:47.033311 20592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:47.033494 20592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "2dc27c2a2d6e4efc865aa0bb506d8a37"
format_stamp: "Formatted at 2026-09-28 14:50:47 on dist-test-slave-vk40"
server_key: "22020c3d61df4f9a0d5f32f7b6cf8d48"
server_key_iv: "a0ff9b4eec1dfdf082010334fa13cf0a"
server_key_version: "encryptionkey@0"
I20260928 14:50:47.033706 20592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:47.048928 20592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:47.049811 20592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:47.050102 20592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:47.067659 20592 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:41935
I20260928 14:50:47.067680 20657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:41935 every 8 connection(s)
I20260928 14:50:47.069078 20592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:47.073276 20658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:47.077109 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 20592
I20260928 14:50:47.077317 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:47.078094 15303 external_mini_cluster.cc:1428] Setting key 082826174bf565b0277518dd9ce5a762
I20260928 14:50:47.086252 20658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37: Bootstrap starting.
I20260928 14:50:47.089551 20658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:47.090808 20658 log.cc:824] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:47.091502 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:47.080087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49950 (local address 127.14.241.254:41935)
0928 14:50:47.081087 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:47.081098 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:47.081139 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:50:47.082764 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:47.082790 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:47.083306 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:47.083711 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:47.084318 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.085622 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.086943 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.087641 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.087827 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:47.088929 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:47.088938 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:47.088954 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:47.089530 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:47.089632 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:47.089656 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:47.089766 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:47.090028 (+ 262us) server_negotiation.cc:299] Negotiation successful
0928 14:50:47.090370 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":343,"threads_started":1}
I20260928 14:50:47.094056 20658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37: No bootstrap required, opened a new log
I20260928 14:50:47.095451 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:50:47.101714 20658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc27c2a2d6e4efc865aa0bb506d8a37" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41935 } }
I20260928 14:50:47.102125 20658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:47.102228 20658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc27c2a2d6e4efc865aa0bb506d8a37, State: Initialized, Role: FOLLOWER
I20260928 14:50:47.102856 20658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc27c2a2d6e4efc865aa0bb506d8a37" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41935 } }
I20260928 14:50:47.103088 20658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:47.103217 20658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:47.103360 20658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:47.105844 20658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc27c2a2d6e4efc865aa0bb506d8a37" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41935 } }
I20260928 14:50:47.106377 20658 leader_election.cc:302] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dc27c2a2d6e4efc865aa0bb506d8a37; no voters:
I20260928 14:50:47.106940 20658 leader_election.cc:288] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:47.107080 20663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:47.107704 20663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [term 1 LEADER]: Becoming Leader. State: Replica: 2dc27c2a2d6e4efc865aa0bb506d8a37, State: Running, Role: LEADER
I20260928 14:50:47.108260 20663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc27c2a2d6e4efc865aa0bb506d8a37" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41935 } }
I20260928 14:50:47.109016 20658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:47.111107 20665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dc27c2a2d6e4efc865aa0bb506d8a37. Latest consensus state: current_term: 1 leader_uuid: "2dc27c2a2d6e4efc865aa0bb506d8a37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc27c2a2d6e4efc865aa0bb506d8a37" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41935 } } }
I20260928 14:50:47.111472 20665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:47.112989 20664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dc27c2a2d6e4efc865aa0bb506d8a37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc27c2a2d6e4efc865aa0bb506d8a37" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41935 } } }
I20260928 14:50:47.113354 20664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:47.115202 20669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:47.122275 20669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:47.130915 20669 catalog_manager.cc:1347] Generated new cluster ID: 15b24d6ffe384b88b257dcc98854b93f
I20260928 14:50:47.131047 20669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:47.182440 20669 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:47.184266 20669 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:47.200801 20669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dc27c2a2d6e4efc865aa0bb506d8a37: Generated new TSK 0
I20260928 14:50:47.201972 20669 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 14:50:47.244705 20689 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260928 14:50:47.244834 20689 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 14:50:47.245761 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:47.217826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.14.241.254:41935)
0928 14:50:47.218027 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:47.218033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:47.218336 (+ 303us) server_negotiation.cc:407] Connection header received
0928 14:50:47.218583 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:47.218587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:47.218664 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:47.218789 (+ 125us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:50:47.219314 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.220070 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.222096 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.222599 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.222691 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:47.223235 (+ 544us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:50:47.245221 (+ 21986us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38847/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 14:50:47.245546 (+ 325us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49952: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38847/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":76}
W20260928 14:50:47.245886 20661 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49952: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38847/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260928 14:50:47.246165 20688 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:50:47.217607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41935 (local address 127.0.0.1:49952)
0928 14:50:47.218232 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:47.218252 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:47.218410 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:47.218825 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:47.218830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:47.218842 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:50:47.219149 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:47.219161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.220199 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.220205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:47.221945 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:47.221962 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.222755 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.222762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:47.222959 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:47.223005 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:50:47.245475 (+ 22470us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38847/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 14:50:47.245760 (+ 285us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.241.254:41935: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38847/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":313,"threads_started":1}
I20260928 14:50:47.247577 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 20592
I20260928 14:50:47.248736 20653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:47.346239 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 20592
2026-09-28T14:50:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (767 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260928 14:50:47.398859 15303 mini_oidc.cc:152] Starting JWKS server
I20260928 14:50:47.400725 15303 webserver.cc:460] Webserver started at https://127.0.0.1:46313/ using document root <none> and password file <none>
I20260928 14:50:47.400933 15303 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 14:50:47.401503 15303 webserver.cc:460] Webserver started at http://127.0.0.1:38961/ using document root <none> and password file <none>
2026-09-28T14:50:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:47Z Disabled control of system clock
I20260928 14:50:47.427127 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:33471
--webserver_interface=127.14.241.254
--webserver_port=0
--builtin_ntp_servers=127.14.241.212:39457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:33471
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46313/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 14:50:47.741712 20696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:47.742029 20696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:47.742163 20696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:47.749387 20696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:47.749511 20696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:47.749576 20696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:47.749658 20696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:47.749718 20696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:50:47.749802 20696 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:50:47.749861 20696 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46313/jwks.json
I20260928 14:50:47.759111 20696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:39457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:33471
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:33471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46313/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:47.760407 20696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:47.762147 20696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:47.773753 20701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:47.773799 20702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:47.773782 20704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:47.774950 20696 server_base.cc:1034] running on GCE node
I20260928 14:50:47.775695 20696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:47.777130 20696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:47.778334 20696 hybrid_clock.cc:648] HybridClock initialized: now 1790607047778287 us; error 31 us; skew 500 ppm
I20260928 14:50:47.778663 20696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:50:47.780844 20696 webserver.cc:460] Webserver started at http://127.14.241.254:32855/ using document root <none> and password file <none>
I20260928 14:50:47.781507 20696 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:47.781603 20696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:47.781901 20696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:47.784737 20696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "3197998f0c064a028d78abb578fed270"
format_stamp: "Formatted at 2026-09-28 14:50:47 on dist-test-slave-vk40"
server_key: "c0ff97944e567b6418ac8572b6afdb56"
server_key_iv: "189927717ca51d287e6cd75d2c8b93db"
server_key_version: "encryptionkey@0"
I20260928 14:50:47.785507 20696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "3197998f0c064a028d78abb578fed270"
format_stamp: "Formatted at 2026-09-28 14:50:47 on dist-test-slave-vk40"
server_key: "c0ff97944e567b6418ac8572b6afdb56"
server_key_iv: "189927717ca51d287e6cd75d2c8b93db"
server_key_version: "encryptionkey@0"
I20260928 14:50:47.790570 20696 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:50:47.793958 20710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:47.795265 20696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:50:47.795437 20696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "3197998f0c064a028d78abb578fed270"
format_stamp: "Formatted at 2026-09-28 14:50:47 on dist-test-slave-vk40"
server_key: "c0ff97944e567b6418ac8572b6afdb56"
server_key_iv: "189927717ca51d287e6cd75d2c8b93db"
server_key_version: "encryptionkey@0"
I20260928 14:50:47.795629 20696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:47.835919 20696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:47.836826 20696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:47.837105 20696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:47.853844 20696 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:33471
I20260928 14:50:47.853821 20761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:33471 every 8 connection(s)
I20260928 14:50:47.855144 20696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:47.858839 20762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:47.864187 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 20696
I20260928 14:50:47.864410 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:47.864980 15303 external_mini_cluster.cc:1428] Setting key ead5bdbe647c514e3286af589c85f17c
I20260928 14:50:47.869706 20762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270: Bootstrap starting.
I20260928 14:50:47.873246 20762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:47.874541 20762 log.cc:824] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:47.877893 20762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270: No bootstrap required, opened a new log
I20260928 14:50:47.879210 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:47.866950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33630 (local address 127.14.241.254:33471)
0928 14:50:47.868022 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:47.868034 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:47.868074 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:50:47.869768 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:47.869797 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:47.870440 (+ 643us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:47.870830 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:47.871394 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.872787 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.874655 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.875659 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.875857 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:47.876950 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:47.876958 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:47.876973 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:47.877510 (+ 537us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:47.877589 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:47.877611 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:47.877694 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:47.877917 (+ 223us) server_negotiation.cc:299] Negotiation successful
0928 14:50:47.878253 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":378,"threads_started":1}
I20260928 14:50:47.883183 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:50:47.886407 20762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3197998f0c064a028d78abb578fed270" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33471 } }
I20260928 14:50:47.886839 20762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:47.886943 20762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3197998f0c064a028d78abb578fed270, State: Initialized, Role: FOLLOWER
I20260928 14:50:47.887470 20762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3197998f0c064a028d78abb578fed270" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33471 } }
I20260928 14:50:47.887688 20762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:47.887809 20762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:47.887969 20762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:47.890563 20762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3197998f0c064a028d78abb578fed270" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33471 } }
I20260928 14:50:47.891076 20762 leader_election.cc:302] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3197998f0c064a028d78abb578fed270; no voters:
I20260928 14:50:47.891628 20762 leader_election.cc:288] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:47.891793 20767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:47.892465 20767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [term 1 LEADER]: Becoming Leader. State: Replica: 3197998f0c064a028d78abb578fed270, State: Running, Role: LEADER
I20260928 14:50:47.893013 20767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3197998f0c064a028d78abb578fed270" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33471 } }
I20260928 14:50:47.893649 20762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:47.895586 20769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3197998f0c064a028d78abb578fed270. Latest consensus state: current_term: 1 leader_uuid: "3197998f0c064a028d78abb578fed270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3197998f0c064a028d78abb578fed270" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33471 } } }
I20260928 14:50:47.895680 20768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3197998f0c064a028d78abb578fed270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3197998f0c064a028d78abb578fed270" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33471 } } }
I20260928 14:50:47.895974 20769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:47.896028 20768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:47.896867 20773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:47.903105 20773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:47.911681 20773 catalog_manager.cc:1347] Generated new cluster ID: c43fc15faabb4f33be82007a65168aa0
I20260928 14:50:47.911803 20773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:47.923528 20773 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:47.925064 20773 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:47.932852 20773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3197998f0c064a028d78abb578fed270: Generated new TSK 0
I20260928 14:50:47.933688 20773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:50:47.954344 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:47.946158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33642 (local address 127.14.241.254:33471)
0928 14:50:47.946344 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:47.946351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:47.946711 (+ 360us) server_negotiation.cc:407] Connection header received
0928 14:50:47.946971 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:47.946974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:47.947051 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:47.947179 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:50:47.947706 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.948473 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.950419 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.950849 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.950955 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:47.951539 (+ 584us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:50:47.953683 (+ 2144us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46313/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 14:50:47.954025 (+ 342us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33642: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46313/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":51}
W20260928 14:50:47.954468 20765 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33642: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46313/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260928 14:50:47.956496 20791 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:50:47.945996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33471 (local address 127.0.0.1:33642)
0928 14:50:47.946598 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:47.946621 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:47.946786 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:47.947220 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:47.947225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:47.947238 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:50:47.947560 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:47.947573 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.948609 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.948615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:47.950270 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:47.950286 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:47.951034 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:47.951041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:47.951244 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:47.951291 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:50:47.953895 (+ 2604us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46313/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 14:50:47.956029 (+ 2134us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.241.254:33471: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46313/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":1804,"thread_start_us":298,"threads_started":1}
I20260928 14:50:47.958393 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 20696
I20260928 14:50:47.959589 20757 generic_service.cc:196] Checking for leaks (request via RPC)
E20260928 14:50:47.965615 20792 webserver.cc:533] Webserver: sslize failed:
E20260928 14:50:47.965708 20792 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 14:50:48.050827 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 20696
2026-09-28T14:50:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (704 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-28T14:50:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:48Z Disabled control of system clock
I20260928 14:50:48.128476 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:43727
--webserver_interface=127.14.241.254
--webserver_port=0
--builtin_ntp_servers=127.14.241.212:33197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:43727
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/testchainca.cert with env {}
W20260928 14:50:48.447566 20797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:48.447857 20797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:48.447942 20797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:48.455154 20797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:48.455276 20797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:48.455339 20797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:48.455430 20797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:48.455489 20797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:50:48.455574 20797 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:50:48.455621 20797 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260928 14:50:48.465610 20797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:33197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:43727
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:43727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:48.466949 20797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:48.468233 20797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:48.479707 20803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:48.479640 20806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:48.479687 20804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:48.485822 20797 server_base.cc:1034] running on GCE node
I20260928 14:50:48.486645 20797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:48.488076 20797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:48.489262 20797 hybrid_clock.cc:648] HybridClock initialized: now 1790607048489208 us; error 56 us; skew 500 ppm
I20260928 14:50:48.489602 20797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:50:48.491899 20797 webserver.cc:460] Webserver started at http://127.14.241.254:34125/ using document root <none> and password file <none>
I20260928 14:50:48.492646 20797 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:48.492754 20797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:48.493042 20797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:48.495875 20797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "9068c322bdef48aa95bf2c3e0d3f91cd"
format_stamp: "Formatted at 2026-09-28 14:50:48 on dist-test-slave-vk40"
server_key: "914e78d8c57e45b90ff36167cf9fdfaf"
server_key_iv: "a849b06aeb7eb7f066e68fa61057c25d"
server_key_version: "encryptionkey@0"
I20260928 14:50:48.496673 20797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "9068c322bdef48aa95bf2c3e0d3f91cd"
format_stamp: "Formatted at 2026-09-28 14:50:48 on dist-test-slave-vk40"
server_key: "914e78d8c57e45b90ff36167cf9fdfaf"
server_key_iv: "a849b06aeb7eb7f066e68fa61057c25d"
server_key_version: "encryptionkey@0"
I20260928 14:50:48.501648 20797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260928 14:50:48.504956 20812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:48.506237 20797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:50:48.506399 20797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "9068c322bdef48aa95bf2c3e0d3f91cd"
format_stamp: "Formatted at 2026-09-28 14:50:48 on dist-test-slave-vk40"
server_key: "914e78d8c57e45b90ff36167cf9fdfaf"
server_key_iv: "a849b06aeb7eb7f066e68fa61057c25d"
server_key_version: "encryptionkey@0"
I20260928 14:50:48.506594 20797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:48.527725 20797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:48.528621 20797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:48.528890 20797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:48.546131 20797 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:43727
I20260928 14:50:48.546149 20863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:43727 every 8 connection(s)
I20260928 14:50:48.547596 20797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:48.551369 20864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:48.557345 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 20797
I20260928 14:50:48.557502 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:48.558094 15303 external_mini_cluster.cc:1428] Setting key bb6452f2ef546f9325d94b4de5b5f585
I20260928 14:50:48.562708 20864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd: Bootstrap starting.
I20260928 14:50:48.566761 20864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:48.567876 20864 log.cc:824] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:48.571115 20864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd: No bootstrap required, opened a new log
I20260928 14:50:48.573266 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:48.560011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60794 (local address 127.14.241.254:43727)
0928 14:50:48.561126 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:48.561141 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:48.561202 (+ 61us) server_negotiation.cc:407] Connection header received
0928 14:50:48.563493 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:48.563523 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:48.564130 (+ 607us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:48.564660 (+ 530us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:48.565234 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:48.567136 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:48.568465 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:48.569241 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:48.569424 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:48.570439 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:48.570454 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:48.570472 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:50:48.571192 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:48.571290 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:50:48.571315 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:48.571426 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:48.571626 (+ 200us) server_negotiation.cc:299] Negotiation successful
0928 14:50:48.571970 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":365,"threads_started":1}
I20260928 14:50:48.576930 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:50:48.579491 20864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9068c322bdef48aa95bf2c3e0d3f91cd" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43727 } }
I20260928 14:50:48.579895 20864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:48.579998 20864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9068c322bdef48aa95bf2c3e0d3f91cd, State: Initialized, Role: FOLLOWER
I20260928 14:50:48.580606 20864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9068c322bdef48aa95bf2c3e0d3f91cd" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43727 } }
I20260928 14:50:48.580847 20864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:48.580963 20864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:48.581107 20864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:48.583531 20864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9068c322bdef48aa95bf2c3e0d3f91cd" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43727 } }
I20260928 14:50:48.584043 20864 leader_election.cc:302] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9068c322bdef48aa95bf2c3e0d3f91cd; no voters:
I20260928 14:50:48.584609 20864 leader_election.cc:288] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:48.584731 20869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:48.585340 20869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [term 1 LEADER]: Becoming Leader. State: Replica: 9068c322bdef48aa95bf2c3e0d3f91cd, State: Running, Role: LEADER
I20260928 14:50:48.585925 20869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9068c322bdef48aa95bf2c3e0d3f91cd" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43727 } }
I20260928 14:50:48.586972 20864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:48.588975 20870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9068c322bdef48aa95bf2c3e0d3f91cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9068c322bdef48aa95bf2c3e0d3f91cd" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43727 } } }
I20260928 14:50:48.589330 20870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:48.588950 20871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9068c322bdef48aa95bf2c3e0d3f91cd. Latest consensus state: current_term: 1 leader_uuid: "9068c322bdef48aa95bf2c3e0d3f91cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9068c322bdef48aa95bf2c3e0d3f91cd" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43727 } } }
I20260928 14:50:48.589552 20871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:48.591042 20875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:48.595853 20875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:48.606873 20875 catalog_manager.cc:1347] Generated new cluster ID: 6ad832d4c91a45b9ba20a8de6b6ce734
I20260928 14:50:48.607007 20875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:48.619911 20875 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:48.621749 20875 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:48.632217 20875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9068c322bdef48aa95bf2c3e0d3f91cd: Generated new TSK 0
I20260928 14:50:48.633277 20875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:50:48.801088 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:48.652873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60796 (local address 127.14.241.254:43727)
0928 14:50:48.653077 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:48.653084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:48.653589 (+ 505us) server_negotiation.cc:407] Connection header received
0928 14:50:48.653841 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:48.653846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:48.653922 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:48.654061 (+ 139us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:50:48.654520 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:48.655256 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:48.656728 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:48.657169 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:48.657276 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:48.657735 (+ 459us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:50:48.800343 (+142608us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 14:50:48.800758 (+ 415us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60796: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":71}
W20260928 14:50:48.801239 20867 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60796: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260928 14:50:48.801317 20893 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:50:48.652754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43727 (local address 127.0.0.1:60796)
0928 14:50:48.653488 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:48.653507 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:48.653650 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:48.654092 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:48.654097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:48.654109 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:50:48.654365 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:48.654374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:48.655404 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:48.655410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:48.656587 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:48.656599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:48.657328 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:48.657334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:48.657463 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:48.657495 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:50:48.800633 (+143138us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 14:50:48.800935 (+ 302us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.241.254:43727: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":389,"threads_started":1}
I20260928 14:50:48.802583 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 20797
I20260928 14:50:48.803762 20859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:48.907673 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 20797
2026-09-28T14:50:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (837 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (410 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (403 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:50:49 dist-test-slave-vk40 krb5kdc[20908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:50:49 dist-test-slave-vk40 krb5kdc[20908](info): set up 2 sockets
Sep 28 14:50:49 dist-test-slave-vk40 krb5kdc[20908](info): commencing operation
krb5kdc: starting...
W20260928 14:50:50.727433 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:50:50 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:50:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:50:50.925869 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:40721
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:40721
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:51.242585 20921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:51.242893 20921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:51.243029 20921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:51.250417 20921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:51.250540 20921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:51.250597 20921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:51.250679 20921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:51.250733 20921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:51.260680 20921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:40721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:40721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:51.262022 20921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:51.263293 20921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:51.274610 20927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:51.274611 20926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:51.274854 20929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:51.275550 20921 server_base.cc:1034] running on GCE node
I20260928 14:50:51.276307 20921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:51.277698 20921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:51.278918 20921 hybrid_clock.cc:648] HybridClock initialized: now 1790607051278858 us; error 38 us; skew 500 ppm
I20260928 14:50:51.279265 20921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:51 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607051, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:51.282274 20921 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:50:51.283763 20921 webserver.cc:460] Webserver started at http://127.14.241.254:33657/ using document root <none> and password file <none>
I20260928 14:50:51.284492 20921 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:51.284610 20921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:51.284899 20921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:51.287833 20921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "c6cc0df45bb14b38b19cf6847b51ad61"
format_stamp: "Formatted at 2026-09-28 14:50:51 on dist-test-slave-vk40"
server_key: "55ded371f26137116de8eeba0bc61595"
server_key_iv: "8e304f625f31ec3c8db1ab05af272db0"
server_key_version: "encryptionkey@0"
I20260928 14:50:51.288630 20921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "c6cc0df45bb14b38b19cf6847b51ad61"
format_stamp: "Formatted at 2026-09-28 14:50:51 on dist-test-slave-vk40"
server_key: "55ded371f26137116de8eeba0bc61595"
server_key_iv: "8e304f625f31ec3c8db1ab05af272db0"
server_key_version: "encryptionkey@0"
I20260928 14:50:51.293720 20921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:50:51.297215 20937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:51.298578 20921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:50:51.298744 20921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "c6cc0df45bb14b38b19cf6847b51ad61"
format_stamp: "Formatted at 2026-09-28 14:50:51 on dist-test-slave-vk40"
server_key: "55ded371f26137116de8eeba0bc61595"
server_key_iv: "8e304f625f31ec3c8db1ab05af272db0"
server_key_version: "encryptionkey@0"
I20260928 14:50:51.298936 20921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:51.315302 20921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:51.319242 20921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:51.319515 20921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:51.336804 20921 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:40721
I20260928 14:50:51.336838 20988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:40721 every 8 connection(s)
I20260928 14:50:51.338142 20921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:51.341696 20989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:51.344442 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 20921
I20260928 14:50:51.344638 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:51.345367 15303 external_mini_cluster.cc:1428] Setting key 7ff4f95bd84b1d3b47c2c49021ec3fbf
I20260928 14:50:51.353931 20989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61: Bootstrap starting.
I20260928 14:50:51.357782 20989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:51.359099 20989 log.cc:824] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61: Log is configured to *not* fsync() on all Append() calls
Sep 28 14:50:51 dist-test-slave-vk40 krb5kdc[20908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:51.362239 20989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61: No bootstrap required, opened a new log
I20260928 14:50:51.368402 20992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:51.347367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.14.241.254:40721)
0928 14:50:51.348073 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:51.348085 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:51.348125 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:50:51.349861 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:51.349887 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:51.350324 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:51.350687 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:51.352423 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:51.353820 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:51.355538 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:51.356232 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:51.356426 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:51.360969 (+ 4543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:51.360977 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:51.360993 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:51.361029 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:51.364069 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:51.364957 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:51.364964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:51.364974 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:51.365165 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:51.365571 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:51.365574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:51.365576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:51.366170 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:51.366391 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:51.366870 (+ 479us) server_negotiation.cc:299] Negotiation successful
0928 14:50:51.367271 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":311,"threads_started":1}
I20260928 14:50:51.370831 20989 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6cc0df45bb14b38b19cf6847b51ad61" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40721 } }
I20260928 14:50:51.371217 20989 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:51.371325 20989 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6cc0df45bb14b38b19cf6847b51ad61, State: Initialized, Role: FOLLOWER
I20260928 14:50:51.371846 20989 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6cc0df45bb14b38b19cf6847b51ad61" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40721 } }
I20260928 14:50:51.372068 20989 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:51.372187 20989 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:51.372372 20989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:51.374562 20989 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6cc0df45bb14b38b19cf6847b51ad61" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40721 } }
I20260928 14:50:51.375046 20989 leader_election.cc:302] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6cc0df45bb14b38b19cf6847b51ad61; no voters:
I20260928 14:50:51.375567 20989 leader_election.cc:288] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:51.375679 20994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:51.376327 20994 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [term 1 LEADER]: Becoming Leader. State: Replica: c6cc0df45bb14b38b19cf6847b51ad61, State: Running, Role: LEADER
I20260928 14:50:51.376869 20994 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6cc0df45bb14b38b19cf6847b51ad61" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40721 } }
I20260928 14:50:51.377525 20989 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:51.379390 20995 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6cc0df45bb14b38b19cf6847b51ad61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6cc0df45bb14b38b19cf6847b51ad61" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40721 } } }
I20260928 14:50:51.379880 20995 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:51.379765 20996 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6cc0df45bb14b38b19cf6847b51ad61. Latest consensus state: current_term: 1 leader_uuid: "c6cc0df45bb14b38b19cf6847b51ad61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6cc0df45bb14b38b19cf6847b51ad61" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40721 } } }
I20260928 14:50:51.380075 20996 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:51.382879 21002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:51.387704 21002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:51.398368 21002 catalog_manager.cc:1347] Generated new cluster ID: 511b0096aa394756b054087a07867031
I20260928 14:50:51.398499 21002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:51.416211 21002 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:51.417557 21002 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:51.424119 21002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6cc0df45bb14b38b19cf6847b51ad61: Generated new TSK 0
I20260928 14:50:51.424983 21002 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:50:51.473016 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:40721
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:51.793509 21015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:51.793809 21015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:51.793929 21015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:51.801476 21015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:51.801594 21015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:51.801743 21015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:51.811921 21015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:40721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:51.813293 21015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:51.814643 21015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:51.827735 21020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:51.827883 21021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:51.828118 21023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:51.828949 21015 server_base.cc:1034] running on GCE node
I20260928 14:50:51.829533 21015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:51.830494 21015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:51.831749 21015 hybrid_clock.cc:648] HybridClock initialized: now 1790607051831684 us; error 47 us; skew 500 ppm
I20260928 14:50:51.832085 21015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:51 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607051, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:51.835248 21015 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:50:51.836802 21015 webserver.cc:460] Webserver started at http://127.14.241.193:33801/ using document root <none> and password file <none>
I20260928 14:50:51.837491 21015 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:51.837610 21015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:51.837909 21015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:51.840941 21015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "68d6183630c1475a8e3d409ef8e34fe2"
format_stamp: "Formatted at 2026-09-28 14:50:51 on dist-test-slave-vk40"
server_key: "f2d071898daea1ed2574e59cbad3d1d4"
server_key_iv: "08350d641ae8d79bdbb03789ef2ad08f"
server_key_version: "encryptionkey@0"
I20260928 14:50:51.841744 21015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "68d6183630c1475a8e3d409ef8e34fe2"
format_stamp: "Formatted at 2026-09-28 14:50:51 on dist-test-slave-vk40"
server_key: "f2d071898daea1ed2574e59cbad3d1d4"
server_key_iv: "08350d641ae8d79bdbb03789ef2ad08f"
server_key_version: "encryptionkey@0"
I20260928 14:50:51.847043 21015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:50:51.850666 21030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:51.852144 21015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:50:51.852320 21015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "68d6183630c1475a8e3d409ef8e34fe2"
format_stamp: "Formatted at 2026-09-28 14:50:51 on dist-test-slave-vk40"
server_key: "f2d071898daea1ed2574e59cbad3d1d4"
server_key_iv: "08350d641ae8d79bdbb03789ef2ad08f"
server_key_version: "encryptionkey@0"
I20260928 14:50:51.852519 21015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:51.880465 21015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:51.884586 21015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:51.884850 21015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:51.885771 21015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:51.887089 21015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:51.887159 21015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:51.887271 21015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:51.887319 21015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:51.916985 21015 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:43075
I20260928 14:50:51.917057 21143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:43075 every 8 connection(s)
I20260928 14:50:51.918267 21015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:51.922264 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 21015
I20260928 14:50:51.922470 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:50:51.923125 15303 external_mini_cluster.cc:1428] Setting key d8fa5ba3a7848bc70f5ecfb690f9fbfe
Sep 28 14:50:51 dist-test-slave-vk40 krb5kdc[20908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607051, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:51.938235 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:51.920710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:33939 (local address 127.14.241.254:40721)
0928 14:50:51.921113 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:51.921118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:51.922224 (+ 1106us) server_negotiation.cc:407] Connection header received
0928 14:50:51.924229 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:51.924235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:51.924361 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:51.924457 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:51.926108 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:51.926892 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:51.928461 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:51.928855 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:51.928963 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:51.932813 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:51.932818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:51.932822 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:51.932856 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:51.935180 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:51.935976 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:51.935979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:51.935982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:51.936094 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:51.936537 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:51.936540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:51.936542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:51.936800 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:51.936921 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:51.937794 (+ 873us) server_negotiation.cc:299] Negotiation successful
0928 14:50:51.937987 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":188,"threads_started":1}
I20260928 14:50:51.939673 21147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:51.921287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:40721 (local address 127.14.241.193:33939)
0928 14:50:51.922031 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:51.922078 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:51.923957 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:51.924722 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:51.924731 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:51.925342 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:51.925943 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:51.925962 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:51.927038 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:51.927042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:51.928321 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:51.928336 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:51.929065 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:51.929072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:51.929299 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:51.929936 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:51.929967 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:51.932586 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:51.935305 (+ 2719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:51.935313 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:51.935330 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:51.935882 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:51.936178 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:51.936183 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:51.936186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:51.936441 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:51.936914 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:51.936922 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:51.937350 (+ 428us) client_negotiation.cc:769] Sending connection context
0928 14:50:51.937666 (+ 316us) client_negotiation.cc:240] Negotiation successful
0928 14:50:51.938232 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":310,"threads_started":1}
I20260928 14:50:51.941175 21144 heartbeater.cc:347] Connected to a master server at 127.14.241.254:40721
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
I20260928 14:50:51.941478 21144 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:51.942171 21144 heartbeater.cc:511] Master 127.14.241.254:40721 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
I20260928 14:50:51.944383 20954 ts_manager.cc:194] Registered new tserver with Master: 68d6183630c1475a8e3d409ef8e34fe2 (127.14.241.193:43075)
I20260928 14:50:51.945833 20954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:33939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:50:51.968353 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:40721
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:52.280122 21150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:52.280437 21150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:52.280539 21150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:52.287833 21150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:52.287949 21150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:52.288069 21150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:52.297626 21150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:40721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:52.298893 21150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:52.300189 21150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:52.312942 21155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:52.313009 21158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:52.312974 21156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:52.313882 21150 server_base.cc:1034] running on GCE node
I20260928 14:50:52.314507 21150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:52.315464 21150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:52.316641 21150 hybrid_clock.cc:648] HybridClock initialized: now 1790607052316589 us; error 42 us; skew 500 ppm
I20260928 14:50:52.317018 21150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:52.319979 21150 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:50:52.321530 21150 webserver.cc:460] Webserver started at http://127.14.241.194:37087/ using document root <none> and password file <none>
I20260928 14:50:52.322247 21150 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:52.322363 21150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:52.322657 21150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:52.325613 21150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "689c40287e004a26b007f5aa81aafa73"
format_stamp: "Formatted at 2026-09-28 14:50:52 on dist-test-slave-vk40"
server_key: "b4da82d8c626b300cc8746fb13c4f820"
server_key_iv: "d4287f39d09093ee38fbe1921a9c6c2b"
server_key_version: "encryptionkey@0"
I20260928 14:50:52.326356 21150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "689c40287e004a26b007f5aa81aafa73"
format_stamp: "Formatted at 2026-09-28 14:50:52 on dist-test-slave-vk40"
server_key: "b4da82d8c626b300cc8746fb13c4f820"
server_key_iv: "d4287f39d09093ee38fbe1921a9c6c2b"
server_key_version: "encryptionkey@0"
I20260928 14:50:52.331297 21150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260928 14:50:52.334664 21165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:52.336015 21150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:50:52.336156 21150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "689c40287e004a26b007f5aa81aafa73"
format_stamp: "Formatted at 2026-09-28 14:50:52 on dist-test-slave-vk40"
server_key: "b4da82d8c626b300cc8746fb13c4f820"
server_key_iv: "d4287f39d09093ee38fbe1921a9c6c2b"
server_key_version: "encryptionkey@0"
I20260928 14:50:52.336403 21150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:52.367708 21150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:52.371861 21150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:52.372128 21150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:52.373080 21150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:52.374354 21150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:52.374425 21150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:52.374536 21150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:52.374583 21150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:52.402707 21150 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:35639
I20260928 14:50:52.402745 21278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:35639 every 8 connection(s)
I20260928 14:50:52.404102 21150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:52.406616 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 21150
I20260928 14:50:52.406804 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:50:52.407312 15303 external_mini_cluster.cc:1428] Setting key 9ef0a8f2ec0c992ae6ad6cd139eed20a
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:52.424347 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:52.406636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:57027 (local address 127.14.241.254:40721)
0928 14:50:52.406805 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:52.406810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:52.408338 (+ 1528us) server_negotiation.cc:407] Connection header received
0928 14:50:52.410636 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:52.410643 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:52.410733 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:52.410820 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:52.412417 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.413150 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.414751 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.415141 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.415261 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:52.418684 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:52.418689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:52.418692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:52.418726 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:52.421211 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:52.421916 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:52.421920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:52.421923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:52.422049 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:52.422475 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:52.422479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:52.422481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:52.422775 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:52.422922 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:52.423834 (+ 912us) server_negotiation.cc:299] Negotiation successful
0928 14:50:52.424028 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
I20260928 14:50:52.425716 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:52.407340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:40721 (local address 127.14.241.194:57027)
0928 14:50:52.408178 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:52.408223 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:52.410345 (+ 2122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:52.411087 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:52.411096 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:52.411628 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:52.412236 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:52.412254 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.413307 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.413313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:52.414599 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:52.414617 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.415360 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.415367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:52.415620 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:52.416362 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:52.416388 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:52.418462 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:52.421346 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:52.421355 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:52.421374 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:52.421789 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:52.422154 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:52.422160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:52.422164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:52.422356 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:52.422932 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:52.422941 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:52.423417 (+ 476us) client_negotiation.cc:769] Sending connection context
0928 14:50:52.423748 (+ 331us) client_negotiation.cc:240] Negotiation successful
0928 14:50:52.424354 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":322,"threads_started":1}
I20260928 14:50:52.427105 21279 heartbeater.cc:347] Connected to a master server at 127.14.241.254:40721
I20260928 14:50:52.427374 21279 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
I20260928 14:50:52.428205 21279 heartbeater.cc:511] Master 127.14.241.254:40721 requested a full tablet report, sending...
I20260928 14:50:52.429805 20954 ts_manager.cc:194] Registered new tserver with Master: 689c40287e004a26b007f5aa81aafa73 (127.14.241.194:35639)
I20260928 14:50:52.430614 20954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:57027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:50:52.451120 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:40721
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:52.764513 21284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:52.764835 21284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:52.764977 21284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:52.772571 21284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:52.772696 21284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:52.772855 21284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:52.782807 21284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:40721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:52.784125 21284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:52.785429 21284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:52.798506 21292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:52.798614 21289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:52.798580 21290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:52.799356 21284 server_base.cc:1034] running on GCE node
I20260928 14:50:52.799974 21284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:52.800940 21284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:52.802134 21284 hybrid_clock.cc:648] HybridClock initialized: now 1790607052802090 us; error 34 us; skew 500 ppm
I20260928 14:50:52.802451 21284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:52.805590 21284 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:50:52.807057 21284 webserver.cc:460] Webserver started at http://127.14.241.195:40163/ using document root <none> and password file <none>
I20260928 14:50:52.807830 21284 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:52.807945 21284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:52.808231 21284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:52.811146 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "133d4be27f1e43999274cdfcb05efa59"
format_stamp: "Formatted at 2026-09-28 14:50:52 on dist-test-slave-vk40"
server_key: "38797dea4e2368e606e56ff8bae10c0f"
server_key_iv: "dcfaac2a4bfdbfa4705666a018855cfa"
server_key_version: "encryptionkey@0"
I20260928 14:50:52.811915 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "133d4be27f1e43999274cdfcb05efa59"
format_stamp: "Formatted at 2026-09-28 14:50:52 on dist-test-slave-vk40"
server_key: "38797dea4e2368e606e56ff8bae10c0f"
server_key_iv: "dcfaac2a4bfdbfa4705666a018855cfa"
server_key_version: "encryptionkey@0"
I20260928 14:50:52.817063 21284 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 14:50:52.820596 21299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:52.821976 21284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:50:52.822118 21284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "133d4be27f1e43999274cdfcb05efa59"
format_stamp: "Formatted at 2026-09-28 14:50:52 on dist-test-slave-vk40"
server_key: "38797dea4e2368e606e56ff8bae10c0f"
server_key_iv: "dcfaac2a4bfdbfa4705666a018855cfa"
server_key_version: "encryptionkey@0"
I20260928 14:50:52.822311 21284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:52.840605 21284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:52.844774 21284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:52.845026 21284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:52.845965 21284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:52.847333 21284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:52.847404 21284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:52.847515 21284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:52.847563 21284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:52.875941 21284 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:42129
I20260928 14:50:52.875979 21412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:42129 every 8 connection(s)
I20260928 14:50:52.877219 21284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:52.879390 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 21284
I20260928 14:50:52.879539 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:50:52.880031 15303 external_mini_cluster.cc:1428] Setting key 125357c0640942cc2ccf45d290cb2625
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:50:52.895860 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:52.879654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:35091 (local address 127.14.241.254:40721)
0928 14:50:52.879882 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:52.879889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:52.881065 (+ 1176us) server_negotiation.cc:407] Connection header received
0928 14:50:52.882964 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:52.882968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:52.883053 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:52.883128 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:52.884580 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.885293 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.886751 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.887137 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.887221 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:52.890435 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:52.890440 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:52.890443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:52.890476 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:52.892909 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:52.893668 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:52.893672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:52.893674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:52.893789 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:52.894199 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:52.894202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:52.894204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:52.894531 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:52.894673 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:52.895443 (+ 770us) server_negotiation.cc:299] Negotiation successful
0928 14:50:52.895608 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 14:50:52.896826 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:52.880120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:40721 (local address 127.14.241.195:35091)
0928 14:50:52.880910 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:52.880955 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:52.882716 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:52.883323 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:52.883330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:52.883821 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:52.884428 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:52.884448 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.885415 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.885418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:52.886637 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:52.886649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.887298 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.887302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:52.887503 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:52.888073 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:52.888098 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:52.890237 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:52.893033 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:52.893042 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:52.893059 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:52.893559 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:52.893892 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:52.893897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:52.893900 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:52.894086 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:52.894691 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:52.894700 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:52.895089 (+ 389us) client_negotiation.cc:769] Sending connection context
0928 14:50:52.895329 (+ 240us) client_negotiation.cc:240] Negotiation successful
0928 14:50:52.895716 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":336,"threads_started":1}
I20260928 14:50:52.898119 21413 heartbeater.cc:347] Connected to a master server at 127.14.241.254:40721
I20260928 14:50:52.898470 21413 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:52.899267 21413 heartbeater.cc:511] Master 127.14.241.254:40721 requested a full tablet report, sending...
I20260928 14:50:52.900699 20954 ts_manager.cc:194] Registered new tserver with Master: 133d4be27f1e43999274cdfcb05efa59 (127.14.241.195:42129)
I20260928 14:50:52.901494 20954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:35091
I20260928 14:50:52.906982 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/test-admin.keytab.
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:52.933821 15303 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260928 14:50:52.938484 21420 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:50:52.936323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.14.241.193:43075)
0928 14:50:52.936920 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:52.936926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:52.936953 (+ 27us) server_negotiation.cc:407] Connection header received
0928 14:50:52.937015 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:52.937022 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:52.937221 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:52.937347 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:52.938104 (+ 757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59330: BlockingRecv error: recv got EOF from 127.0.0.1:59330 (error 108)
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":322,"threads_started":1}
W20260928 14:50:52.938530 15303 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/krb5cc' of type FILE
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:52.940240 15303 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:52.941617 15303 init.cc:295] Successfully reacquired a new kerberos TGT
I20260928 14:50:52.948731 21144 heartbeater.cc:503] Master 127.14.241.254:40721 was elected leader, sending a full tablet report...
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:52.956439 21420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:52.943097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.14.241.193:43075)
0928 14:50:52.943279 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:52.943285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:52.943521 (+ 236us) server_negotiation.cc:407] Connection header received
0928 14:50:52.943664 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:52.943668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:52.943729 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:52.943800 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:52.944746 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.945694 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.946852 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.947427 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.947515 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:52.951050 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:52.951062 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:52.951076 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:52.951109 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:52.953808 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:52.954448 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:52.954455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:52.954457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:52.954571 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:52.954927 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:52.954932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:52.954935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:52.955282 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:52.955555 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:52.955849 (+ 294us) server_negotiation.cc:299] Negotiation successful
0928 14:50:52.956121 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260928 14:50:52.961694 21420 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:50:52.960225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.14.241.193:43075)
0928 14:50:52.960434 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:52.960439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:52.960579 (+ 140us) server_negotiation.cc:407] Connection header received
0928 14:50:52.960697 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:52.960701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:52.960756 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:52.960827 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:52.961474 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59342: BlockingRecv error: recv got EOF from 127.0.0.1:59342 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260928 14:50:52.961855 15303 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/krb5cc' of type FILE
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:52.963217 15303 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:52.964411 15303 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:52.976805 21420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:52.965686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.14.241.193:43075)
0928 14:50:52.965874 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:52.965880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:52.965996 (+ 116us) server_negotiation.cc:407] Connection header received
0928 14:50:52.966116 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:52.966119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:52.966178 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:52.966248 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:52.967125 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.967823 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.968976 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.969382 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.969476 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:52.972537 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:52.972542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:52.972545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:52.972586 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:52.974864 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:52.975362 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:52.975366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:52.975368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:52.975475 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:52.975728 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:52.975730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:52.975732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:52.975983 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:52.976163 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:52.976366 (+ 203us) server_negotiation.cc:299] Negotiation successful
0928 14:50:52.976519 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260928 14:50:52.980729 21420 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:50:52.979195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.14.241.193:43075)
0928 14:50:52.979351 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:52.979357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:52.979581 (+ 224us) server_negotiation.cc:407] Connection header received
0928 14:50:52.979728 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:52.979732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:52.979797 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:52.979870 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:52.980531 (+ 661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59350: BlockingRecv error: recv got EOF from 127.0.0.1:59350 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260928 14:50:52.980897 15303 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790607004819300-15303-0/krb5kdc/krb5cc' of type FILE
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:52.982380 15303 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:52.983600 15303 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 14:50:52 dist-test-slave-vk40 krb5kdc[20908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:50:52.996978 21420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:52.984746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59364 (local address 127.14.241.193:43075)
0928 14:50:52.984941 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:52.984946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:52.985175 (+ 229us) server_negotiation.cc:407] Connection header received
0928 14:50:52.985318 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:52.985323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:52.985386 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:52.985465 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:52.986380 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.987110 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.988325 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:52.988738 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:52.988844 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:52.992044 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:52.992049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:52.992052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:52.992086 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:52.994538 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:52.995154 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:52.995158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:52.995161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:52.995309 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:52.995645 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:52.995649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:52.995651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:52.995926 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:52.996207 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:52.996470 (+ 263us) server_negotiation.cc:299] Negotiation successful
0928 14:50:52.996674 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:50:52.998473 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 21015
I20260928 14:50:53.009181 21420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:52.999158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59380 (local address 127.14.241.193:43075)
0928 14:50:52.999308 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:52.999312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:52.999613 (+ 301us) server_negotiation.cc:407] Connection header received
0928 14:50:52.999746 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:52.999749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:52.999805 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:52.999874 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:53.000712 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:53.001360 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:53.002500 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:53.002930 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:53.003016 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:53.004854 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:53.004860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:53.004863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:53.004894 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:53.007185 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:53.007707 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:53.007710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:53.007712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:53.007836 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:53.008086 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:53.008088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:53.008090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:53.008368 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:53.008556 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:53.008772 (+ 216us) server_negotiation.cc:299] Negotiation successful
0928 14:50:53.008925 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:50:53.009678 21139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:53.091723 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 21015
I20260928 14:50:53.106127 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 21150
Sep 28 14:50:53 dist-test-slave-vk40 krb5kdc[20908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:50:53.121954 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:53.106925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.14.241.194:35639)
0928 14:50:53.107464 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:53.107471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:53.107495 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:50:53.107555 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:53.107561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:53.107764 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:53.107893 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:53.108751 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:53.109780 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:53.110966 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:53.111778 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:53.111910 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:53.116684 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:53.116702 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:53.116722 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:53.116768 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:53.119438 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:53.120067 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:53.120074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:53.120076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:53.120194 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:53.120608 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:53.120612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:53.120614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:53.120978 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:53.121217 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:53.121457 (+ 240us) server_negotiation.cc:299] Negotiation successful
0928 14:50:53.121656 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":272,"threads_started":1}
I20260928 14:50:53.123247 21274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:53.204175 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 21150
I20260928 14:50:53.216956 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 21284
Sep 28 14:50:53 dist-test-slave-vk40 krb5kdc[20908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:50:53.231920 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:53.217863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40390 (local address 127.14.241.195:42129)
0928 14:50:53.218457 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:53.218464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:53.218491 (+ 27us) server_negotiation.cc:407] Connection header received
0928 14:50:53.218551 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:53.218557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:53.218760 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:53.218921 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:53.219937 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:53.220963 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:53.222227 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:53.222768 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:53.222885 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:53.226628 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:53.226639 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:53.226655 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:53.226718 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:53.229289 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:53.229966 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:53.229973 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:53.229976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:53.230103 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:53.230444 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:53.230448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:53.230450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:53.230792 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:53.231101 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:53.231356 (+ 255us) server_negotiation.cc:299] Negotiation successful
0928 14:50:53.231604 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":309,"threads_started":1}
I20260928 14:50:53.233246 21408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:53.316319 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 21284
I20260928 14:50:53.328771 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 20921
I20260928 14:50:53.329840 20984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:53.432909 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 20921
2026-09-28T14:50:53Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3715 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:50:53 dist-test-slave-vk40 krb5kdc[21440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:50:53 dist-test-slave-vk40 krb5kdc[21440](info): set up 2 sockets
Sep 28 14:50:53 dist-test-slave-vk40 krb5kdc[21440](info): commencing operation
krb5kdc: starting...
W20260928 14:50:54.439743 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.969s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:50:54 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:50:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:50:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.254.keytab.
Entry for principal oryx/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.254.keytab.
I20260928 14:50:54.639657 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:33413
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.254.keytab
--principal=oryx/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:33413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:54.957167 21453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:54.957466 21453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:54.957598 21453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:54.964906 21453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:54.965031 21453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:54.965106 21453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:54.965181 21453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:54.965240 21453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:54.975515 21453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:33413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.254.keytab
--principal=oryx/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:33413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:54.976890 21453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:54.978185 21453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:54.989377 21458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:54.989435 21459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:54.989405 21461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:54.990115 21453 server_base.cc:1034] running on GCE node
I20260928 14:50:54.990998 21453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:54.992460 21453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:54.993655 21453 hybrid_clock.cc:648] HybridClock initialized: now 1790607054993606 us; error 41 us; skew 500 ppm
I20260928 14:50:54.994019 21453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:54 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607054, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:54.996960 21453 init.cc:377] Logged in from keytab as oryx/127.14.241.254@KRBTEST.COM (short username oryx)
I20260928 14:50:54.998354 21453 webserver.cc:460] Webserver started at http://127.14.241.254:36907/ using document root <none> and password file <none>
I20260928 14:50:54.999002 21453 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:54.999119 21453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:54.999405 21453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:55.002369 21453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "9917d61e0e9d413b839dfc49f84766a4"
format_stamp: "Formatted at 2026-09-28 14:50:54 on dist-test-slave-vk40"
server_key: "6cff266d927f681e7db66ccc4ea21b92"
server_key_iv: "5bc214403260b18d2e5ed2c1c438534d"
server_key_version: "encryptionkey@0"
I20260928 14:50:55.003142 21453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "9917d61e0e9d413b839dfc49f84766a4"
format_stamp: "Formatted at 2026-09-28 14:50:54 on dist-test-slave-vk40"
server_key: "6cff266d927f681e7db66ccc4ea21b92"
server_key_iv: "5bc214403260b18d2e5ed2c1c438534d"
server_key_version: "encryptionkey@0"
I20260928 14:50:55.008265 21453 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260928 14:50:55.011634 21468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:55.013047 21453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260928 14:50:55.013209 21453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "9917d61e0e9d413b839dfc49f84766a4"
format_stamp: "Formatted at 2026-09-28 14:50:54 on dist-test-slave-vk40"
server_key: "6cff266d927f681e7db66ccc4ea21b92"
server_key_iv: "5bc214403260b18d2e5ed2c1c438534d"
server_key_version: "encryptionkey@0"
I20260928 14:50:55.013391 21453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:55.038729 21453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:55.042668 21453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:55.042948 21453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:55.059969 21453 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:33413
I20260928 14:50:55.059998 21520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:33413 every 8 connection(s)
I20260928 14:50:55.061328 21453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:50:55.065266 21521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:55.068979 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 21453
I20260928 14:50:55.069233 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:50:55.070043 15303 external_mini_cluster.cc:1428] Setting key 46d50c47b8554234579c46e6648831b8
I20260928 14:50:55.079173 21521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4: Bootstrap starting.
I20260928 14:50:55.083194 21521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:55.084550 21521 log.cc:824] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4: Log is configured to *not* fsync() on all Append() calls
Sep 28 14:50:55 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:50:55.087586 21521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4: No bootstrap required, opened a new log
I20260928 14:50:55.093104 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:55.072461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56234 (local address 127.14.241.254:33413)
0928 14:50:55.073321 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:55.073333 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:55.073381 (+ 48us) server_negotiation.cc:407] Connection header received
0928 14:50:55.075010 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:55.075033 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:55.075472 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:55.075836 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:55.077591 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:55.078901 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:55.080661 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:55.081359 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:55.081497 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:55.086156 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:55.086164 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:55.086179 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:55.086222 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:55.089159 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:55.089820 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:55.089827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:55.089837 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:55.090023 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:55.090371 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:55.090379 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:55.090381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:55.090899 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:55.091181 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:55.091627 (+ 446us) server_negotiation.cc:299] Negotiation successful
0928 14:50:55.092034 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":399,"threads_started":1}
I20260928 14:50:55.095693 21521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } }
I20260928 14:50:55.096113 21521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:55.096213 21521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9917d61e0e9d413b839dfc49f84766a4, State: Initialized, Role: FOLLOWER
I20260928 14:50:55.096730 21521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } }
I20260928 14:50:55.096942 21521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:50:55.097033 21521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:50:55.097150 21521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:55.099149 21521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } }
I20260928 14:50:55.099581 21521 leader_election.cc:302] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9917d61e0e9d413b839dfc49f84766a4; no voters:
I20260928 14:50:55.100251 21521 leader_election.cc:288] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:50:55.100415 21526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:50:55.101007 21526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 1 LEADER]: Becoming Leader. State: Replica: 9917d61e0e9d413b839dfc49f84766a4, State: Running, Role: LEADER
I20260928 14:50:55.101641 21526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } }
I20260928 14:50:55.103624 21528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } }
I20260928 14:50:55.104072 21528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:55.103670 21527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9917d61e0e9d413b839dfc49f84766a4. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } }
I20260928 14:50:55.104614 21527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:55.104676 21521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:55.109128 21530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:50:55.115500 21530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:50:55.126375 21530 catalog_manager.cc:1347] Generated new cluster ID: a0ac33f33ad647ed9c41bab5190b2b4d
I20260928 14:50:55.126490 21530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:50:55.136916 21530 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:50:55.138818 21530 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:50:55.150638 21530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4: Generated new TSK 0
I20260928 14:50:55.151795 21530 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab.
Entry for principal oryx/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab.
I20260928 14:50:55.204489 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33413
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab
--principal=oryx/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:55.525192 21547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:55.525470 21547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:55.525588 21547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:55.532894 21547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:55.533004 21547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:55.533151 21547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:55.543085 21547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab
--principal=oryx/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:55.544395 21547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:55.545725 21547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:55.558429 21552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:55.558350 21553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:55.558383 21555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:55.559422 21547 server_base.cc:1034] running on GCE node
I20260928 14:50:55.560026 21547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:55.560966 21547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:55.562172 21547 hybrid_clock.cc:648] HybridClock initialized: now 1790607055562112 us; error 59 us; skew 500 ppm
I20260928 14:50:55.562481 21547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:55 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607055, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:55.571358 21547 init.cc:377] Logged in from keytab as oryx/127.14.241.193@KRBTEST.COM (short username oryx)
I20260928 14:50:55.572906 21547 webserver.cc:460] Webserver started at http://127.14.241.193:46501/ using document root <none> and password file <none>
I20260928 14:50:55.573593 21547 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:55.573712 21547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:55.573998 21547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:55.576901 21547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "aaef801444514d3eb59008afc430a19a"
format_stamp: "Formatted at 2026-09-28 14:50:55 on dist-test-slave-vk40"
server_key: "2a6ce618ffa9bd6f34d4281cb68cab39"
server_key_iv: "baabaa0dd968f54759124e3420974afd"
server_key_version: "encryptionkey@0"
I20260928 14:50:55.577636 21547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "aaef801444514d3eb59008afc430a19a"
format_stamp: "Formatted at 2026-09-28 14:50:55 on dist-test-slave-vk40"
server_key: "2a6ce618ffa9bd6f34d4281cb68cab39"
server_key_iv: "baabaa0dd968f54759124e3420974afd"
server_key_version: "encryptionkey@0"
I20260928 14:50:55.582582 21547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:50:55.585958 21562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:55.587256 21547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:50:55.587410 21547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "aaef801444514d3eb59008afc430a19a"
format_stamp: "Formatted at 2026-09-28 14:50:55 on dist-test-slave-vk40"
server_key: "2a6ce618ffa9bd6f34d4281cb68cab39"
server_key_iv: "baabaa0dd968f54759124e3420974afd"
server_key_version: "encryptionkey@0"
I20260928 14:50:55.587580 21547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:55.611799 21547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:55.616115 21547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:55.616418 21547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:55.617355 21547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:55.618659 21547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:55.618728 21547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:55.618839 21547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:55.618893 21547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:55.646986 21547 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:44073
I20260928 14:50:55.647037 21675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:44073 every 8 connection(s)
I20260928 14:50:55.648213 21547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:55.653985 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 21547
I20260928 14:50:55.654173 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:50:55.654758 15303 external_mini_cluster.cc:1428] Setting key 0046cc32d58397451efe02369ca68113
Sep 28 14:50:55 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607055, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:50:55.668030 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:55.650664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:35583 (local address 127.14.241.254:33413)
0928 14:50:55.651105 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:55.651110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:55.652063 (+ 953us) server_negotiation.cc:407] Connection header received
0928 14:50:55.654043 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:55.654049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:55.654137 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:55.654240 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:55.655905 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:55.656862 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:55.658421 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:55.658987 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:55.659103 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:55.662606 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:55.662612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:55.662615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:55.662649 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:55.665026 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:55.665837 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:55.665840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:55.665843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:55.665964 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:55.666384 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:55.666387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:55.666389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:55.666642 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:55.666760 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:55.667581 (+ 821us) server_negotiation.cc:299] Negotiation successful
0928 14:50:55.667753 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":339,"thread_start_us":247,"threads_started":1}
I20260928 14:50:55.669521 21679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:55.651216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.193:35583)
0928 14:50:55.651908 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:55.651951 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:55.653761 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:55.654487 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:55.654496 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:55.655042 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:55.655737 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:55.655764 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:55.657021 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:55.657025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:55.658257 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:55.658271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:55.659172 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:55.659176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:55.659371 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:55.660011 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:55.660033 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:55.662353 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:55.665179 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:55.665189 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:55.665207 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:55.665739 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:55.666050 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:55.666055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:55.666059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:55.666293 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:55.666764 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:55.666772 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:55.667198 (+ 426us) client_negotiation.cc:769] Sending connection context
0928 14:50:55.667498 (+ 300us) client_negotiation.cc:240] Negotiation successful
0928 14:50:55.668043 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":252,"threads_started":1}
I20260928 14:50:55.671042 21676 heartbeater.cc:347] Connected to a master server at 127.14.241.254:33413
I20260928 14:50:55.671382 21676 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:55.672339 21676 heartbeater.cc:511] Master 127.14.241.254:33413 requested a full tablet report, sending...
I20260928 14:50:55.674633 21486 ts_manager.cc:194] Registered new tserver with Master: aaef801444514d3eb59008afc430a19a (127.14.241.193:44073)
WARNING: no policy specified for oryx/127.14.241.194@KRBTEST.COM; defaulting to no policy
I20260928 14:50:55.676100 21486 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:35583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab.
Entry for principal oryx/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab.
I20260928 14:50:55.700758 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33413
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab
--principal=oryx/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:56.016544 21682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:56.016842 21682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:56.016975 21682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:56.024384 21682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:56.024497 21682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:56.024610 21682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:56.034718 21682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab
--principal=oryx/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:56.035975 21682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:56.037297 21682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:56.052773 21687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:56.052819 21688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:56.052850 21690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:56.053646 21682 server_base.cc:1034] running on GCE node
I20260928 14:50:56.054229 21682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:56.055154 21682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:56.056371 21682 hybrid_clock.cc:648] HybridClock initialized: now 1790607056056316 us; error 46 us; skew 500 ppm
I20260928 14:50:56.056671 21682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:56 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607056, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:56.059688 21682 init.cc:377] Logged in from keytab as oryx/127.14.241.194@KRBTEST.COM (short username oryx)
I20260928 14:50:56.061170 21682 webserver.cc:460] Webserver started at http://127.14.241.194:38145/ using document root <none> and password file <none>
I20260928 14:50:56.061846 21682 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:56.061959 21682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:56.062232 21682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:56.065122 21682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "c22174d815bd4fd29ffb608bef8e7786"
format_stamp: "Formatted at 2026-09-28 14:50:56 on dist-test-slave-vk40"
server_key: "fbd046596bf7af2ef5e4e78114c3aebc"
server_key_iv: "b77916a6859c379350a845a127a00e41"
server_key_version: "encryptionkey@0"
I20260928 14:50:56.065897 21682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "c22174d815bd4fd29ffb608bef8e7786"
format_stamp: "Formatted at 2026-09-28 14:50:56 on dist-test-slave-vk40"
server_key: "fbd046596bf7af2ef5e4e78114c3aebc"
server_key_iv: "b77916a6859c379350a845a127a00e41"
server_key_version: "encryptionkey@0"
I20260928 14:50:56.071404 21682 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260928 14:50:56.074956 21697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:56.076323 21682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 14:50:56.076491 21682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "c22174d815bd4fd29ffb608bef8e7786"
format_stamp: "Formatted at 2026-09-28 14:50:56 on dist-test-slave-vk40"
server_key: "fbd046596bf7af2ef5e4e78114c3aebc"
server_key_iv: "b77916a6859c379350a845a127a00e41"
server_key_version: "encryptionkey@0"
I20260928 14:50:56.076681 21682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:56.094087 21682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:56.098537 21682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:56.098806 21682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:56.099756 21682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:56.101205 21682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:56.101284 21682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:56.101444 21682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:56.101498 21682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:56.132115 21682 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:39459
I20260928 14:50:56.132150 21810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:39459 every 8 connection(s)
I20260928 14:50:56.133421 21682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:56.139142 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 21682
I20260928 14:50:56.139338 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:50:56.139882 15303 external_mini_cluster.cc:1428] Setting key d1fa6c7341dd8504dfcecdab3ee98496
Sep 28 14:50:56 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607056, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:50:56.153450 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:56.135819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:43883 (local address 127.14.241.254:33413)
0928 14:50:56.135989 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:56.135993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:56.137150 (+ 1157us) server_negotiation.cc:407] Connection header received
0928 14:50:56.138894 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:56.138899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:56.138976 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:56.139074 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:56.141149 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:56.141961 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:56.143523 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:56.143930 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:56.144034 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:56.147735 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:56.147741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:56.147744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:56.147780 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:56.150385 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:56.151247 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:56.151250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:56.151253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:56.151393 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:56.151811 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:56.151814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:56.151816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:56.152077 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:56.152184 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:56.153007 (+ 823us) server_negotiation.cc:299] Negotiation successful
0928 14:50:56.153183 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:50:56.155336 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:56.136255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.194:43883)
0928 14:50:56.136984 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:56.137025 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:56.138616 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:56.139439 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:56.139452 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:56.140196 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:56.140955 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:56.140982 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:56.142109 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:56.142113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:56.143382 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:56.143394 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:56.144142 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:56.144149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:56.144424 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:56.145051 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:56.145072 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:56.147468 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:56.150539 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:56.150548 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:56.150568 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:56.151122 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:56.151484 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:56.151488 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:56.151491 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:56.151717 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:56.152186 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:56.152193 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:56.152638 (+ 445us) client_negotiation.cc:769] Sending connection context
0928 14:50:56.152924 (+ 286us) client_negotiation.cc:240] Negotiation successful
0928 14:50:56.153488 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":264,"threads_started":1}
I20260928 14:50:56.156924 21811 heartbeater.cc:347] Connected to a master server at 127.14.241.254:33413
I20260928 14:50:56.157259 21811 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:56.158020 21811 heartbeater.cc:511] Master 127.14.241.254:33413 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.14.241.195@KRBTEST.COM; defaulting to no policy
I20260928 14:50:56.159502 21486 ts_manager.cc:194] Registered new tserver with Master: c22174d815bd4fd29ffb608bef8e7786 (127.14.241.194:39459)
I20260928 14:50:56.160336 21486 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:43883
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab.
Entry for principal oryx/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab.
I20260928 14:50:56.186822 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33413
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab
--principal=oryx/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:56.500241 21816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:56.500559 21816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:56.500689 21816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:56.508216 21816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:56.508379 21816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:56.508535 21816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:56.518457 21816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab
--principal=oryx/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:56.519768 21816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:56.521111 21816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:56.533668 21824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:56.533663 21822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:56.533849 21821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:56.534538 21816 server_base.cc:1034] running on GCE node
I20260928 14:50:56.535079 21816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:56.535950 21816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:56.537138 21816 hybrid_clock.cc:648] HybridClock initialized: now 1790607056537089 us; error 38 us; skew 500 ppm
I20260928 14:50:56.537439 21816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:56 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607056, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:56.540532 21816 init.cc:377] Logged in from keytab as oryx/127.14.241.195@KRBTEST.COM (short username oryx)
I20260928 14:50:56.541931 21816 webserver.cc:460] Webserver started at http://127.14.241.195:44793/ using document root <none> and password file <none>
I20260928 14:50:56.542605 21816 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:56.542718 21816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:56.543001 21816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:56.546048 21816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "b0fc3d05546d4301baae6c6202709f82"
format_stamp: "Formatted at 2026-09-28 14:50:56 on dist-test-slave-vk40"
server_key: "db9f633cd38cb1d00d300ab6052fcd1b"
server_key_iv: "18647a7371710153a424b4caab467f22"
server_key_version: "encryptionkey@0"
I20260928 14:50:56.546808 21816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "b0fc3d05546d4301baae6c6202709f82"
format_stamp: "Formatted at 2026-09-28 14:50:56 on dist-test-slave-vk40"
server_key: "db9f633cd38cb1d00d300ab6052fcd1b"
server_key_iv: "18647a7371710153a424b4caab467f22"
server_key_version: "encryptionkey@0"
I20260928 14:50:56.551828 21816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 14:50:56.555208 21831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:56.556620 21816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:56.556763 21816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "b0fc3d05546d4301baae6c6202709f82"
format_stamp: "Formatted at 2026-09-28 14:50:56 on dist-test-slave-vk40"
server_key: "db9f633cd38cb1d00d300ab6052fcd1b"
server_key_iv: "18647a7371710153a424b4caab467f22"
server_key_version: "encryptionkey@0"
I20260928 14:50:56.556948 21816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:56.574092 21816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:56.578651 21816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:56.578933 21816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:56.579879 21816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:56.581378 21816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:56.581456 21816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:56.581581 21816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:56.581630 21816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:56.610939 21816 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:42739
I20260928 14:50:56.610985 21944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:42739 every 8 connection(s)
I20260928 14:50:56.612545 21816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:56.615446 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 21816
I20260928 14:50:56.615587 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:50:56.615983 15303 external_mini_cluster.cc:1428] Setting key f1b54916f9a69bfa271a209c2f05e731
Sep 28 14:50:56 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607056, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:50:56.631390 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:56.614933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:50533 (local address 127.14.241.254:33413)
0928 14:50:56.615133 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:56.615140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:56.616478 (+ 1338us) server_negotiation.cc:407] Connection header received
0928 14:50:56.618336 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:56.618342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:56.618424 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:56.618523 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:56.619963 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:56.620739 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:56.622191 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:56.622605 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:56.622689 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:56.625851 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:56.625856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:56.625859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:56.625899 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:56.628415 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:56.629164 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:56.629170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:56.629174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:56.629300 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:56.629704 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:56.629708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:56.629719 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:56.629997 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:56.630128 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:56.630830 (+ 702us) server_negotiation.cc:299] Negotiation successful
0928 14:50:56.631052 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 14:50:56.632444 21947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:56.615487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.195:50533)
0928 14:50:56.616299 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:56.616344 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:56.618066 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:56.618717 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:56.618726 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:56.619224 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:56.619826 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:56.619844 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:56.620852 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:56.620856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:56.622086 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:56.622098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:56.622775 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:56.622781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:56.622984 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:56.623607 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:56.623632 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:56.625646 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:56.628563 (+ 2917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:56.628570 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:56.628600 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:56.629035 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:56.629408 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:56.629411 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:56.629413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:56.629598 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:56.630120 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:56.630128 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:56.630484 (+ 356us) client_negotiation.cc:769] Sending connection context
0928 14:50:56.630729 (+ 245us) client_negotiation.cc:240] Negotiation successful
0928 14:50:56.631269 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":301,"threads_started":1}
I20260928 14:50:56.633697 21945 heartbeater.cc:347] Connected to a master server at 127.14.241.254:33413
I20260928 14:50:56.633980 21945 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:56.634677 21945 heartbeater.cc:511] Master 127.14.241.254:33413 requested a full tablet report, sending...
I20260928 14:50:56.636034 21486 ts_manager.cc:194] Registered new tserver with Master: b0fc3d05546d4301baae6c6202709f82 (127.14.241.195:42739)
I20260928 14:50:56.636849 21486 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:50533
I20260928 14:50:56.642910 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.241.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.253@KRBTEST.COM" created.
I20260928 14:50:56.679163 21676 heartbeater.cc:503] Master 127.14.241.254:33413 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.253.keytab.
Entry for principal oryx/127.14.241.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.253.keytab.
I20260928 14:50:56.686046 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.253:35531
--webserver_interface=127.14.241.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.253.keytab
--principal=oryx/127.14.241.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:33413,127.14.241.253:35531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:56.999792 21950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:57.000109 21950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:57.000247 21950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:57.007680 21950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:57.007810 21950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:57.007889 21950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:57.007997 21950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:57.008059 21950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:57.018095 21950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:33413,127.14.241.253:35531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.253.keytab
--principal=oryx/127.14.241.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.253:35531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.241.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
W20260928 14:50:57.018508 21950 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.14.241.254:33413,127.14.241.253:35531'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260928 14:50:57.019541 21950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:57.020846 21950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:57.032130 21955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:57.032107 21956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:57.032164 21958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:57.033114 21950 server_base.cc:1034] running on GCE node
I20260928 14:50:57.033833 21950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:57.035197 21950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:57.036376 21950 hybrid_clock.cc:648] HybridClock initialized: now 1790607057036297 us; error 49 us; skew 500 ppm
I20260928 14:50:57.036679 21950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607057, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:57.039642 21950 init.cc:377] Logged in from keytab as oryx/127.14.241.253@KRBTEST.COM (short username oryx)
I20260928 14:50:57.041196 21950 webserver.cc:460] Webserver started at http://127.14.241.253:38163/ using document root <none> and password file <none>
I20260928 14:50:57.041889 21950 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:57.041989 21950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:57.042277 21950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:57.045298 21950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data/instance:
uuid: "86d4b41acb1a441c87a2f0794cb6ea53"
format_stamp: "Formatted at 2026-09-28 14:50:57 on dist-test-slave-vk40"
server_key: "a1da16eb9c1fd4c5d73df65fab4dfc0f"
server_key_iv: "817bcca5af146185a94f92570e2eaac0"
server_key_version: "encryptionkey@0"
I20260928 14:50:57.046069 21950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/wal/instance:
uuid: "86d4b41acb1a441c87a2f0794cb6ea53"
format_stamp: "Formatted at 2026-09-28 14:50:57 on dist-test-slave-vk40"
server_key: "a1da16eb9c1fd4c5d73df65fab4dfc0f"
server_key_iv: "817bcca5af146185a94f92570e2eaac0"
server_key_version: "encryptionkey@0"
I20260928 14:50:57.051245 21950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:50:57.054714 21965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:57.056027 21950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 14:50:57.056174 21950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/wal
uuid: "86d4b41acb1a441c87a2f0794cb6ea53"
format_stamp: "Formatted at 2026-09-28 14:50:57 on dist-test-slave-vk40"
server_key: "a1da16eb9c1fd4c5d73df65fab4dfc0f"
server_key_iv: "817bcca5af146185a94f92570e2eaac0"
server_key_version: "encryptionkey@0"
I20260928 14:50:57.056449 21950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:57.087968 21950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:57.092520 21950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:57.092783 21950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:57.115041 21950 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.253:35531
I20260928 14:50:57.115080 22016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.253:35531 every 8 connection(s)
I20260928 14:50:57.116842 21950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
I20260928 14:50:57.121362 22017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:57.125502 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 21950
I20260928 14:50:57.129060 22017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.241.253@KRBTEST.COM
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607057, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.253@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:50:57.150735 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.127697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42302 (local address 127.14.241.253:35531)
0928 14:50:57.128521 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.128538 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.128588 (+ 50us) server_negotiation.cc:407] Connection header received
0928 14:50:57.130290 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.130321 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.130896 (+ 575us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.131349 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.133249 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.135206 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.137058 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.138074 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.138260 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.143183 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.143194 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.143210 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.143257 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.146564 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.147254 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.147263 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.147273 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.147426 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.147800 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.147803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.147806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.148345 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.148631 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.149065 (+ 434us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.149457 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":329,"threads_started":1}
I20260928 14:50:57.152629 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.133029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56244)
0928 14:50:57.139013 (+ 5984us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.139058 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.139206 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.139619 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.139626 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.140529 (+ 903us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.140861 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.140875 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.142019 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.142022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.143235 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.143247 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.143951 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.143957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.144151 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.144721 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.144743 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.147178 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.150003 (+ 2825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.150012 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.150018 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.150541 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.150885 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.150890 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.150894 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.151077 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.151641 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.151648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.151830 (+ 182us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.152041 (+ 211us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.152320 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5716,"thread_start_us":5570,"threads_started":1}
I20260928 14:50:57.152859 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.133330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56244 (local address 127.14.241.254:33413)
0928 14:50:57.138460 (+ 5130us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.138465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.139147 (+ 682us) server_negotiation.cc:407] Connection header received
0928 14:50:57.139353 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.139359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.139435 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.139524 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.141031 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.141857 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.143373 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.143777 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.143912 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.147343 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.147348 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.147351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.147387 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.149875 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.150648 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.150652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.150654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.150778 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.151202 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.151206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.151209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.151525 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.151637 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.152186 (+ 549us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.152454 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4997,"thread_start_us":4875,"threads_started":1}
I20260928 14:50:57.154368 22017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607057, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.253@KRBTEST.COM for oryx/127.14.241.253@KRBTEST.COM
I20260928 14:50:57.163219 21811 heartbeater.cc:503] Master 127.14.241.254:33413 was elected leader, sending a full tablet report...
I20260928 14:50:57.166167 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.155084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:35531 (local address 127.0.0.1:42306)
0928 14:50:57.155219 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.155233 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.155315 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.155546 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.155549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.155937 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.156077 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.156084 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.157049 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.157053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.157888 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.157898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.158530 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.158534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.158681 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.159210 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.159226 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.161276 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.164015 (+ 2739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.164019 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.164022 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.164517 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.164849 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.164854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.164858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.165011 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.165480 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.165484 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.165603 (+ 119us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.165706 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.165918 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 14:50:57.166311 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.155031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.14.241.253:35531)
0928 14:50:57.155195 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.155201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.155318 (+ 117us) server_negotiation.cc:407] Connection header received
0928 14:50:57.155388 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.155392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.155447 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.155520 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.156195 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.156923 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.158001 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.158404 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.158504 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.161451 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.161456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.161459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.161486 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.163906 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.164621 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.164626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.164628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.164765 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.165103 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.165107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.165109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.165372 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.165487 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.165769 (+ 282us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.166031 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":50}
I20260928 14:50:57.170742 22017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: Bootstrap starting.
I20260928 14:50:57.173795 22017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:57.175017 22017 log.cc:824] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:57.177762 22017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: No bootstrap required, opened a new log
I20260928 14:50:57.184749 22017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } }
I20260928 14:50:57.185125 22017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:57.185215 22017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d4b41acb1a441c87a2f0794cb6ea53, State: Initialized, Role: FOLLOWER
I20260928 14:50:57.185808 22017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } }
I20260928 14:50:57.186729 22025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } } }
I20260928 14:50:57.187090 22025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:50:57.187875 22017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 14:50:57.195732 22036 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:50:57.195855 22036 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:50:57.198855 21950 master_runner.cc:428] Detected that this master 86d4b41acb1a441c87a2f0794cb6ea53 is joining an existing cluster
I20260928 14:50:57.198971 21950 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.241.253:35531
I20260928 14:50:57.219625 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.203492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56248 (local address 127.14.241.254:33413)
0928 14:50:57.203654 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.203660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.204782 (+ 1122us) server_negotiation.cc:407] Connection header received
0928 14:50:57.205058 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.205064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.205164 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.205250 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.206780 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.207890 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.210173 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.210858 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.210967 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.213675 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.213681 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.213686 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.213725 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.216916 (+ 3191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.217645 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.217650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.217654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.217809 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.218248 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.218251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.218254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.218529 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.218661 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.219075 (+ 414us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.219289 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:50:57.220050 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.203725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56248)
0928 14:50:57.204669 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.204693 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.204896 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.205486 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.205490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.206152 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.206575 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.206590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.208081 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.208086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.209944 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.210009 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.211301 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.211309 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.211508 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.212243 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.212267 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.213453 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.217030 (+ 3577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.217036 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.217040 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.217504 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.217949 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.217953 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.217957 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.218127 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.218642 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.218647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.218820 (+ 173us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.218953 (+ 133us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.219267 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":306,"threads_started":1}
I20260928 14:50:57.229655 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.203428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.14.241.253:35531)
0928 14:50:57.203653 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.203659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.213887 (+ 10228us) server_negotiation.cc:407] Connection header received
0928 14:50:57.214078 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.214083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.214152 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.214211 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.215520 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.216668 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.219926 (+ 3258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.220641 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.220749 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.222987 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.222993 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.222997 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.223027 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.226646 (+ 3619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.227441 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.227446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.227449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.227598 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.227987 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.227992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.227995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.228398 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.228548 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.229033 (+ 485us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.229268 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:50:57.230736 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.204526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:35531 (local address 127.0.0.1:42322)
0928 14:50:57.213794 (+ 9268us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.213822 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.213970 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.214383 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.214388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.215040 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.215348 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.215361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.216869 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.216875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.218556 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.218570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.220997 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.221003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.221155 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.221798 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.221819 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.222798 (+ 979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.226831 (+ 4033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.226836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.226840 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.227331 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.227712 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.227716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.227719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.227874 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.228628 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.228632 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.228784 (+ 152us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.229974 (+ 1190us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.230284 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9111,"thread_start_us":7413,"threads_started":1}
WARNING: no policy specified for oryx/127.14.241.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.252@KRBTEST.COM" created.
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607054, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.254@KRBTEST.COM for oryx/127.14.241.253@KRBTEST.COM
I20260928 14:50:57.258818 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.228472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.14.241.253:35531)
0928 14:50:57.229937 (+ 1465us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.229942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.245178 (+ 15236us) server_negotiation.cc:407] Connection header received
0928 14:50:57.245501 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.245506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.245585 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.245687 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.246965 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.247701 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.249329 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.250031 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.250159 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.253289 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.253294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.253297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.253328 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.256049 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.256772 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.256780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.256783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.256913 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.257276 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.257280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.257282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.257600 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.257726 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.258219 (+ 493us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.258448 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1367,"thread_start_us":17172,"threads_started":1}
I20260928 14:50:57.258850 22045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.228012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:35531 (local address 127.0.0.1:42334)
0928 14:50:57.244972 (+ 16960us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.245026 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.245293 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.245777 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.245786 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.246475 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.246796 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.246808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.247825 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.247829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.249153 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.249169 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.250203 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.250208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.250340 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.250953 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.250970 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.253112 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.256176 (+ 3064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.256190 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.256198 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.256652 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.257014 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.257019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.257023 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.257170 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.257726 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.257735 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.257929 (+ 194us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.258096 (+ 167us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.258424 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16620,"thread_start_us":16430,"threads_started":1}
I20260928 14:50:57.261111 21486 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.241.253:35531: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53"
member_type: NON_VOTER
last_known_addr {
host: "127.14.241.253"
port: 35531
}
attrs {
promote: true
}
}
dest_uuid: "9917d61e0e9d413b839dfc49f84766a4"
cas_config_opid_index: -1
I20260928 14:50:57.262146 21486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } }
I20260928 14:50:57.265694 22049 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 86d4b41acb1a441c87a2f0794cb6ea53 (127.14.241.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } } }
I20260928 14:50:57.266911 21992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:50:57.270412 22049 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.241.253:35531
I20260928 14:50:57.270467 21992 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 FOLLOWER]: Refusing update from remote peer 9917d61e0e9d413b839dfc49f84766a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260928 14:50:57.270982 22049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } } }
I20260928 14:50:57.271201 21950 master.cc:537] Master@127.14.241.253:35531 shutting down...
I20260928 14:50:57.271353 22049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:57.271935 22050 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.252.keytab.
Entry for principal oryx/127.14.241.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.252.keytab.
W20260928 14:50:57.275276 21472 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260928 14:50:57.275646 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.252:42347
--webserver_interface=127.14.241.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.252.keytab
--principal=oryx/127.14.241.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:33413,127.14.241.253:35531,127.14.241.252:42347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:50:57.286000 21950 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:57.286473 21950 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:57.286692 21950 tablet_replica.cc:330] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: stopping tablet replica
W20260928 14:50:57.287293 21472 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 -> Peer 86d4b41acb1a441c87a2f0794cb6ea53 (127.14.241.253:35531): Couldn't send request to peer 86d4b41acb1a441c87a2f0794cb6ea53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 14:50:57.293572 21950 master.cc:552] Master@127.14.241.253:35531 shutdown complete.
I20260928 14:50:57.293664 21950 master_runner.cc:305] Clearing existing system tablet
I20260928 14:50:57.294082 21950 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:57.294173 21950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:57.297178 21950 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260928 14:50:57.304715 22055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:57.305141 21950 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:57.305421 21950 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.007s
I20260928 14:50:57.305513 21950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/wal
uuid: "86d4b41acb1a441c87a2f0794cb6ea53"
format_stamp: "Formatted at 2026-09-28 14:50:57 on dist-test-slave-vk40"
server_key: "a1da16eb9c1fd4c5d73df65fab4dfc0f"
server_key_iv: "817bcca5af146185a94f92570e2eaac0"
server_key_version: "encryptionkey@0"
I20260928 14:50:57.310650 21950 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:57.317685 21950 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 14:50:57.317889 21950 log.cc:1196] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260928 14:50:57.318310 21950 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: Deleting consensus metadata
I20260928 14:50:57.318568 21950 master_runner.cc:315] Copying system tablet from 127.14.241.254:33413
I20260928 14:50:57.321036 21950 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: tablet copy: Beginning tablet copy session from remote peer at address 127.14.241.254:33413
I20260928 14:50:57.339071 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.321870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56258 (local address 127.14.241.254:33413)
0928 14:50:57.322071 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.322077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.324826 (+ 2749us) server_negotiation.cc:407] Connection header received
0928 14:50:57.325063 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.325070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.325158 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.325273 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.326317 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.327490 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.329361 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.330001 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.330097 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.332539 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.332546 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.332551 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.332597 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.336163 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.336957 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.336962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.336966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.337116 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.337518 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.337523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.337526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.337881 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.338031 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.338500 (+ 469us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.338718 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 14:50:57.339157 22061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.321808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56258)
0928 14:50:57.324682 (+ 2874us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.324702 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.324877 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.325302 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.325308 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.325854 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.326140 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.326159 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.327645 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.327651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.329182 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.329198 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.330363 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.330369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.330552 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.331243 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.331267 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.332266 (+ 999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.336317 (+ 4051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.336323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.336328 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.336819 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.337221 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.337226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.337230 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.337399 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.338012 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.338017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.338209 (+ 192us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.338364 (+ 155us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.338707 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2724,"thread_start_us":2629,"threads_started":1}
I20260928 14:50:57.340378 21506 tablet_copy_service.cc:140] P 9917d61e0e9d413b839dfc49f84766a4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 86d4b41acb1a441c87a2f0794cb6ea53 ({username='oryx', principal='oryx/127.14.241.253@KRBTEST.COM'} at 127.0.0.1:56258)
I20260928 14:50:57.340651 21506 tablet_copy_service.cc:161] P 9917d61e0e9d413b839dfc49f84766a4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 86d4b41acb1a441c87a2f0794cb6ea53 at {username='oryx', principal='oryx/127.14.241.253@KRBTEST.COM'} at 127.0.0.1:56258: session id = 86d4b41acb1a441c87a2f0794cb6ea53-00000000000000000000000000000000
I20260928 14:50:57.344928 21506 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 14:50:57.347203 21950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 14:50:57.353870 21950 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: tablet copy: Starting download of 0 data blocks...
I20260928 14:50:57.354189 21950 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: tablet copy: Starting download of 1 WAL segments...
I20260928 14:50:57.357240 21950 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 14:50:57.361740 21506 tablet_copy_service.cc:342] P 9917d61e0e9d413b839dfc49f84766a4: Request end of tablet copy session 86d4b41acb1a441c87a2f0794cb6ea53-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.241.253@KRBTEST.COM'} at 127.0.0.1:56258
I20260928 14:50:57.361975 21506 tablet_copy_service.cc:433] P 9917d61e0e9d413b839dfc49f84766a4: ending tablet copy session 86d4b41acb1a441c87a2f0794cb6ea53-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 86d4b41acb1a441c87a2f0794cb6ea53
I20260928 14:50:57.374397 21950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:57.377137 22062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:57.377970 22063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:57.388536 21950 server_base.cc:1034] running on GCE node
W20260928 14:50:57.389603 22065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:57.390118 21950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:57.390935 21950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:57.392217 21950 hybrid_clock.cc:648] HybridClock initialized: now 1790607057392145 us; error 38 us; skew 500 ppm
I20260928 14:50:57.392510 21950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607057, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:57.394631 21950 init.cc:377] Logged in from keytab as oryx/127.14.241.253@KRBTEST.COM (short username oryx)
I20260928 14:50:57.395900 21950 webserver.cc:460] Webserver started at http://127.14.241.253:43833/ using document root <none> and password file <none>
I20260928 14:50:57.396330 21950 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:57.396425 21950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:57.399523 21950 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260928 14:50:57.401506 22072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:57.401969 21950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:50:57.402108 21950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/wal
uuid: "86d4b41acb1a441c87a2f0794cb6ea53"
format_stamp: "Formatted at 2026-09-28 14:50:57 on dist-test-slave-vk40"
server_key: "a1da16eb9c1fd4c5d73df65fab4dfc0f"
server_key_iv: "817bcca5af146185a94f92570e2eaac0"
server_key_version: "encryptionkey@0"
I20260928 14:50:57.402264 21950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:57.436239 21950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:57.440480 21950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:57.467633 21950 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.253:35531
I20260928 14:50:57.467662 22123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.253:35531 every 8 connection(s)
I20260928 14:50:57.468657 21950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
I20260928 14:50:57.474246 22124 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:50:57.478071 22124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: Bootstrap starting.
I20260928 14:50:57.495023 22124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:50:57.495532 22124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: Bootstrap complete.
I20260928 14:50:57.496596 22124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } }
I20260928 14:50:57.496856 22124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 86d4b41acb1a441c87a2f0794cb6ea53, State: Initialized, Role: LEARNER
I20260928 14:50:57.497097 22124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } }
I20260928 14:50:57.497892 22127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } } }
I20260928 14:50:57.498162 22124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:57.498396 22127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: This master's current role is: LEARNER
I20260928 14:50:57.512583 22138 catalog_manager.cc:1259] Loaded cluster ID: a0ac33f33ad647ed9c41bab5190b2b4d
I20260928 14:50:57.512706 22138 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: loading cluster ID for follower catalog manager: success
I20260928 14:50:57.516422 22138 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: acquiring CA information for follower catalog manager: success
I20260928 14:50:57.519052 22138 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:50:57.639374 21945 heartbeater.cc:503] Master 127.14.241.254:33413 was elected leader, sending a full tablet report...
W20260928 14:50:57.654395 22053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:57.654692 22053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:57.654826 22053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:57.663014 22053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:50:57.663287 22053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:57.663406 22053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:57.663491 22053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:50:57.663551 22053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:50:57.674494 22053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:33413,127.14.241.253:35531,127.14.241.252:42347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.252.keytab
--principal=oryx/127.14.241.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.252:42347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.241.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:57.676167 22053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:57.677835 22053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:57.692162 22146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:57.692350 22144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:57.692430 22143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:57.693814 22053 server_base.cc:1034] running on GCE node
I20260928 14:50:57.694573 22053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:57.696108 22053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:57.697387 22053 hybrid_clock.cc:648] HybridClock initialized: now 1790607057697275 us; error 59 us; skew 500 ppm
I20260928 14:50:57.697880 22053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607057, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:57.700997 22053 init.cc:377] Logged in from keytab as oryx/127.14.241.252@KRBTEST.COM (short username oryx)
I20260928 14:50:57.702536 22053 webserver.cc:460] Webserver started at http://127.14.241.252:45477/ using document root <none> and password file <none>
I20260928 14:50:57.703214 22053 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:57.703346 22053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:57.703665 22053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:50:57.707177 22053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data/instance:
uuid: "d852e652ec6343c69c1e315766983c7b"
format_stamp: "Formatted at 2026-09-28 14:50:57 on dist-test-slave-vk40"
server_key: "f96f07605158e71307f6f97c5c86070d"
server_key_iv: "64f2548a288d269291e04fed4f1bec38"
server_key_version: "encryptionkey@0"
I20260928 14:50:57.707999 22053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/wal/instance:
uuid: "d852e652ec6343c69c1e315766983c7b"
format_stamp: "Formatted at 2026-09-28 14:50:57 on dist-test-slave-vk40"
server_key: "f96f07605158e71307f6f97c5c86070d"
server_key_iv: "64f2548a288d269291e04fed4f1bec38"
server_key_version: "encryptionkey@0"
I20260928 14:50:57.713757 22053 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260928 14:50:57.717150 22153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:57.718497 22053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:50:57.718638 22053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/wal
uuid: "d852e652ec6343c69c1e315766983c7b"
format_stamp: "Formatted at 2026-09-28 14:50:57 on dist-test-slave-vk40"
server_key: "f96f07605158e71307f6f97c5c86070d"
server_key_iv: "64f2548a288d269291e04fed4f1bec38"
server_key_version: "encryptionkey@0"
I20260928 14:50:57.718807 22053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:57.739717 22053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:57.743899 22053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:57.744144 22053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:57.772119 22053 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.252:42347
I20260928 14:50:57.772154 22205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.252:42347 every 8 connection(s)
I20260928 14:50:57.773502 22053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
I20260928 14:50:57.774190 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 22053
I20260928 14:50:57.778208 22206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:57.788511 22206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.241.252@KRBTEST.COM
I20260928 14:50:57.799814 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.776543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45670 (local address 127.14.241.252:42347)
0928 14:50:57.777468 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.777485 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.777777 (+ 292us) server_negotiation.cc:407] Connection header received
0928 14:50:57.779976 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.780004 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.780535 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.780986 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.782379 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.784482 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.785747 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.786820 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.786999 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.792187 (+ 5188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.792198 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.792217 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.792258 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.795675 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.796380 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.796388 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.796397 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.796591 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.796918 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.796923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.796927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.797507 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.797745 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.798253 (+ 508us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.798642 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":404,"threads_started":1}
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607057, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.252@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:50:57.806892 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.791035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.14.241.254:33413)
0928 14:50:57.791195 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.791199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.792078 (+ 879us) server_negotiation.cc:407] Connection header received
0928 14:50:57.792275 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.792311 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.792390 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.792466 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.794816 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.795535 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.797181 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.797574 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.797685 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.801335 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.801339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.801342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.801383 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.804012 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.804771 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.804778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.804781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.804933 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.805333 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.805338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.805342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.805614 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.805749 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.806312 (+ 563us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.806531 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:50:57.807193 22210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.791058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56272)
0928 14:50:57.791935 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.791979 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.792154 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.792562 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.792569 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.794060 (+ 1491us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.794651 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.794667 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.795676 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.795681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.797054 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.797066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.797757 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.797764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.797934 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.798593 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.798609 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.801141 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.804130 (+ 2989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.804142 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.804148 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.804638 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.805036 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.805041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.805045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.805236 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.805735 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.805742 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.806022 (+ 280us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.806183 (+ 161us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.806523 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":417,"threads_started":1}
I20260928 14:50:57.808463 22206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607057, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.252@KRBTEST.COM for oryx/127.14.241.253@KRBTEST.COM
I20260928 14:50:57.823938 22210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.809148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:35531 (local address 127.0.0.1:42344)
0928 14:50:57.809387 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.809406 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.809505 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.810046 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.810051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.810522 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.810772 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.810781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.812268 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.812272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.813694 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.813707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.814608 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.814614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.814803 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.815461 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.815477 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.817746 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.821279 (+ 3533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.821285 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.821289 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.821793 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.822227 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.822232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.822236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.822402 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.822953 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.822957 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.823119 (+ 162us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.823237 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.823561 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 14:50:57.823982 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.809162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.14.241.253:35531)
0928 14:50:57.809719 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.809723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.809738 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:50:57.809816 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.809821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.809896 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.809988 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.810921 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.812101 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.813855 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.814439 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.814553 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.817943 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.817948 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.817951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.817984 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.821157 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.821903 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.821908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.821911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.822123 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.822499 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.822503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.822505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.822845 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.823026 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.823335 (+ 309us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.823597 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":279,"threads_started":1}
I20260928 14:50:57.825268 22206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607057, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.252@KRBTEST.COM for oryx/127.14.241.252@KRBTEST.COM
I20260928 14:50:57.839241 22210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.825914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42347 (local address 127.0.0.1:45680)
0928 14:50:57.826162 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.826176 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.826260 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.826486 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.826489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.827061 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.827292 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.827301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.828245 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.828249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.829183 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.829193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.829817 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.829821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.829944 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.830416 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.830431 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.832828 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.835977 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.835982 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.835987 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.836538 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.837002 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.837007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.837010 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.837298 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.837936 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.837940 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.838212 (+ 272us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.838350 (+ 138us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.838625 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 14:50:57.839638 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.825969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.14.241.252:42347)
0928 14:50:57.826108 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.826112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.826225 (+ 113us) server_negotiation.cc:407] Connection header received
0928 14:50:57.826342 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.826345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.826402 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.826462 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.827403 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.828150 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.829306 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.829693 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.829781 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.833063 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.833067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.833071 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.833106 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.835823 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.836649 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.836654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.836657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.836890 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.837426 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.837429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.837431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.837821 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.837947 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.838487 (+ 540us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.838892 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":49}
I20260928 14:50:57.844655 22206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Bootstrap starting.
I20260928 14:50:57.847764 22206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:57.849324 22206 log.cc:824] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:57.852550 22206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: No bootstrap required, opened a new log
I20260928 14:50:57.862480 22206 raft_consensus.cc:348] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } }
I20260928 14:50:57.862885 22206 raft_consensus.cc:374] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:57.862978 22206 raft_consensus.cc:729] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d852e652ec6343c69c1e315766983c7b, State: Initialized, Role: FOLLOWER
I20260928 14:50:57.863516 22206 consensus_queue.cc:260] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } }
I20260928 14:50:57.864354 22213 sys_catalog.cc:455] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } } }
I20260928 14:50:57.864697 22213 sys_catalog.cc:458] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:50:57.865607 22206 sys_catalog.cc:564] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: configured and running, proceeding with master startup.
W20260928 14:50:57.877125 22224 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:50:57.877247 22224 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:50:57.880834 22053 master_runner.cc:428] Detected that this master d852e652ec6343c69c1e315766983c7b is joining an existing cluster
I20260928 14:50:57.880971 22053 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.241.252:42347
I20260928 14:50:57.893271 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.884338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.14.241.253:35531)
0928 14:50:57.884529 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.884535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.886014 (+ 1479us) server_negotiation.cc:407] Connection header received
0928 14:50:57.886219 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.886225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.886380 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.886496 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:57.886963 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.888217 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.890773 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.892066 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.892244 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.892489 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.892812 (+ 323us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.892981 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 14:50:57.894075 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.884219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:35531 (local address 127.0.0.1:42350)
0928 14:50:57.885890 (+ 1671us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.885912 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.886078 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.886544 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.886550 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.886574 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:57.886820 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.886831 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.888378 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.888384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.890573 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.890595 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.892307 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.892315 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.892577 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.892653 (+ 76us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.893337 (+ 684us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.893738 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1536,"thread_start_us":1392,"threads_started":1}
I20260928 14:50:57.895344 22099 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260928 14:50:57.895990 22127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9917d61e0e9d413b839dfc49f84766a4. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } } }
I20260928 14:50:57.896319 22127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: This master's current role is: LEARNER
I20260928 14:50:57.908356 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 21547
I20260928 14:50:57.922858 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.897605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.14.241.253:35531)
0928 14:50:57.897759 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.897764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.899712 (+ 1948us) server_negotiation.cc:407] Connection header received
0928 14:50:57.899927 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.899931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.900001 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.900061 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.903377 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.905211 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.908080 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.908935 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.909035 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.912832 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.912840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.912846 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.912891 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.917462 (+ 4571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.918465 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.918470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.918474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.918713 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.919336 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.919341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.919344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.919716 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.919840 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.921963 (+ 2123us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.922326 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:50:57.925482 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.897422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56280 (local address 127.14.241.254:33413)
0928 14:50:57.897574 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.897578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.899481 (+ 1903us) server_negotiation.cc:407] Connection header received
0928 14:50:57.899734 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.899739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.899814 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.899890 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.902094 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.903727 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.909922 (+ 6195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.910844 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.910972 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.914981 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.914988 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.914993 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.915034 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.918657 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.919632 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.919637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.919640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.919901 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.920679 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.920682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.920684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.920944 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.921075 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.923952 (+ 2877us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.924977 (+ 1025us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:50:57.925730 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.897913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:35531 (local address 127.0.0.1:42358)
0928 14:50:57.899639 (+ 1726us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.899655 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.899793 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.900239 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.900243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.902006 (+ 1763us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.903156 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.903173 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.905535 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.905542 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.907898 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.907915 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.909546 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.909553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.909890 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.910883 (+ 993us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.910909 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.912561 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.917729 (+ 5168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.917735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.917739 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.918319 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.918877 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.918881 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.918884 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.919187 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.919936 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.919940 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.920484 (+ 544us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.923156 (+ 2672us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.923694 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1551,"thread_start_us":1344,"threads_started":1}
I20260928 14:50:57.928813 22232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.897665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56280)
0928 14:50:57.899368 (+ 1703us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.899387 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.899573 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.900105 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.900110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.901181 (+ 1071us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.901922 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.901937 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.904092 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.904098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.906699 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.906716 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.912258 (+ 5542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.912267 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.912590 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.913437 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.913462 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.914724 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.918811 (+ 4087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.918817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.918821 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.919477 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.920041 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.920046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.920049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.920361 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.921065 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.921069 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.923325 (+ 2256us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.923452 (+ 127us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.927330 (+ 3878us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1542,"mutex_wait_us":1061,"thread_start_us":1141,"threads_started":1}
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.241.193@KRBTEST.COM
I20260928 14:50:57.940544 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.900046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45688 (local address 127.14.241.252:42347)
0928 14:50:57.900269 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.900275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.917581 (+ 17306us) server_negotiation.cc:407] Connection header received
0928 14:50:57.917730 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.917735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.917816 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.917913 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.921603 (+ 3690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.922801 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.928368 (+ 5567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.929237 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.929359 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.932537 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.932543 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.932547 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.932580 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.937053 (+ 4473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.937916 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.937921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.937924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.938132 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.938568 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.938573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.938576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.939038 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.939196 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.939778 (+ 582us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.939994 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:50:57.942481 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.909694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34740 (local address 127.14.241.193:44073)
0928 14:50:57.912149 (+ 2455us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.912160 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.912192 (+ 32us) server_negotiation.cc:407] Connection header received
0928 14:50:57.912261 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.912271 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.912617 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.916388 (+ 3771us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.916759 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.921157 (+ 4398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.923493 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.924582 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.925383 (+ 801us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.931674 (+ 6291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.931692 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.931710 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.931752 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.936879 (+ 5127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.938081 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.938092 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.938096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.938391 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.939020 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.939026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.939029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.939521 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.941311 (+ 1790us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.941510 (+ 199us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.941808 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2248,"thread_start_us":2048,"threads_started":1}
I20260928 14:50:57.942641 22234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.900419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42347 (local address 127.0.0.1:45688)
0928 14:50:57.917479 (+ 17060us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.917499 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.917635 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.918154 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.918159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.919588 (+ 1429us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.921490 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.921503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.923095 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.923100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.928192 (+ 5092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.928210 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.929725 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.929733 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.929943 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.930642 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.930662 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.932341 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.937207 (+ 4866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.937212 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.937216 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.937804 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.938270 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.938274 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.938277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.938455 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.939275 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.939279 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.939481 (+ 202us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.941327 (+ 1846us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.941727 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16906,"thread_start_us":17183,"threads_started":1}
I20260928 14:50:57.944146 21671 generic_service.cc:196] Checking for leaks (request via RPC)
Sep 28 14:50:57 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607054, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.254@KRBTEST.COM for oryx/127.14.241.252@KRBTEST.COM
I20260928 14:50:57.960615 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.935605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42347 (local address 127.0.0.1:45694)
0928 14:50:57.935973 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:57.935992 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:57.936097 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:57.941202 (+ 5105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:57.941209 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:57.941818 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:57.942105 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.942119 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.944371 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.944376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.945861 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:57.945877 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.947654 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.947660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:57.947874 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.948666 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:57.948689 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:57.952479 (+ 3790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:57.957314 (+ 4835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.957320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.957325 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.958070 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.958559 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:57.958564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:57.958568 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:57.958782 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:57.959459 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:57.959464 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:57.959647 (+ 183us) client_negotiation.cc:769] Sending connection context
0928 14:50:57.959817 (+ 170us) client_negotiation.cc:240] Negotiation successful
0928 14:50:57.960122 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260928 14:50:57.961616 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:57.936320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45694 (local address 127.14.241.252:42347)
0928 14:50:57.940795 (+ 4475us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:57.940801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:57.940818 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:50:57.940904 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:57.940908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:57.941049 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:57.941181 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:57.942296 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.944187 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.946040 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:57.947439 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:57.947626 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:57.952697 (+ 5071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:57.952704 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:57.952709 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:57.952746 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:57.957154 (+ 4408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.958204 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.958210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.958214 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.958420 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:57.958889 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:57.958893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:57.958897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:57.959318 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:57.959464 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:57.959941 (+ 477us) server_negotiation.cc:299] Negotiation successful
0928 14:50:57.960167 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4364,"thread_start_us":8010,"threads_started":1}
I20260928 14:50:57.963476 21486 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.241.252:42347: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d852e652ec6343c69c1e315766983c7b"
member_type: NON_VOTER
last_known_addr {
host: "127.14.241.252"
port: 42347
}
attrs {
promote: true
}
}
dest_uuid: "9917d61e0e9d413b839dfc49f84766a4"
cas_config_opid_index: 5
I20260928 14:50:57.964696 21486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } }
I20260928 14:50:57.966790 22226 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d852e652ec6343c69c1e315766983c7b (127.14.241.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
I20260928 14:50:57.971197 22226 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.241.252:42347
I20260928 14:50:57.972000 22053 master.cc:537] Master@127.14.241.252:42347 shutting down...
I20260928 14:50:57.974007 22099 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 LEARNER]: Refusing update from remote peer 9917d61e0e9d413b839dfc49f84766a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260928 14:50:57.986646 22226 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260928 14:50:57.992990 22238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
I20260928 14:50:57.993464 22238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: This master's current role is: LEADER
W20260928 14:50:58.002324 21472 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 -> Peer d852e652ec6343c69c1e315766983c7b (127.14.241.252:42347): Couldn't send request to peer d852e652ec6343c69c1e315766983c7b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 14:50:58.006532 22099 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d852e652ec6343c69c1e315766983c7b (127.14.241.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
I20260928 14:50:58.012249 22127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: NON_VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: true } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
I20260928 14:50:58.012600 22127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: This master's current role is: LEARNER
I20260928 14:50:58.013764 22226 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4: attempting to promote NON_VOTER 86d4b41acb1a441c87a2f0794cb6ea53 to VOTER
I20260928 14:50:58.014714 22226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } }
W20260928 14:50:58.018666 21472 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 -> Peer d852e652ec6343c69c1e315766983c7b (127.14.241.252:42347): Couldn't send request to peer d852e652ec6343c69c1e315766983c7b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 14:50:58.020087 22098 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 LEARNER]: Refusing update from remote peer 9917d61e0e9d413b839dfc49f84766a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260928 14:50:58.021049 22238 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260928 14:50:58.024897 22239 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 86d4b41acb1a441c87a2f0794cb6ea53 (127.14.241.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
W20260928 14:50:58.029383 22159 tablet.cc:2369] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:50:58.031952 22053 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260928 14:50:58.032400 22053 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260928 14:50:58.032588 22053 tablet_replica.cc:330] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: stopping tablet replica
I20260928 14:50:58.032917 22098 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 86d4b41acb1a441c87a2f0794cb6ea53 (127.14.241.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
I20260928 14:50:58.037720 22127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
I20260928 14:50:58.038026 22127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:50:58.039811 22238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
I20260928 14:50:58.039748 22239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
I20260928 14:50:58.040050 22238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:58.040083 22239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:58.049091 22053 master.cc:552] Master@127.14.241.252:42347 shutdown complete.
I20260928 14:50:58.049183 22053 master_runner.cc:305] Clearing existing system tablet
I20260928 14:50:58.049506 22053 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:58.049621 22053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:58.052040 22053 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260928 14:50:58.053431 22246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:58.053763 22053 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:58.054036 22053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:50:58.054133 22053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/wal
uuid: "d852e652ec6343c69c1e315766983c7b"
format_stamp: "Formatted at 2026-09-28 14:50:57 on dist-test-slave-vk40"
server_key: "f96f07605158e71307f6f97c5c86070d"
server_key_iv: "64f2548a288d269291e04fed4f1bec38"
server_key_version: "encryptionkey@0"
I20260928 14:50:58.057687 22053 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:50:58.063892 22053 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 14:50:58.064093 22053 log.cc:1196] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260928 14:50:58.064694 22053 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Deleting consensus metadata
I20260928 14:50:58.064980 22053 master_runner.cc:315] Copying system tablet from 127.14.241.254:33413
I20260928 14:50:58.085325 22053 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: tablet copy: Beginning tablet copy session from remote peer at address 127.14.241.254:33413
I20260928 14:50:58.087247 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 21547
I20260928 14:50:58.104184 22252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:58.086121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56290)
0928 14:50:58.087046 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:58.087071 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:58.087230 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:58.089208 (+ 1978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:58.089214 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:58.089794 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:58.090106 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:58.090120 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.091572 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.091577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:58.093290 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:58.093316 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.094448 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.094455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:58.094662 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:58.095508 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:58.095538 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:58.096798 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:58.101107 (+ 4309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:58.101113 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:58.101122 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:58.101654 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:58.102150 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:58.102156 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:58.102160 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:58.102345 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:58.103084 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:58.103089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:58.103286 (+ 197us) client_negotiation.cc:769] Sending connection context
0928 14:50:58.103418 (+ 132us) client_negotiation.cc:240] Negotiation successful
0928 14:50:58.103784 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":653,"threads_started":1}
I20260928 14:50:58.104842 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:44073
--local_ip_for_outbound_sockets=127.14.241.193
--tserver_master_addrs=127.14.241.254:33413
--webserver_port=46501
--webserver_interface=127.14.241.193
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab
--principal=oryx/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.241.254:33413,127.14.241.253:35531
--tserver_master_addrs=127.14.241.254:33413,127.14.241.253:35531,127.14.241.252:42347 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:50:58.105024 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:58.086132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.14.241.254:33413)
0928 14:50:58.086373 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:58.086380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:58.087178 (+ 798us) server_negotiation.cc:407] Connection header received
0928 14:50:58.087364 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:58.087369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:58.087456 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:58.087548 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:58.090307 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.091393 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.093526 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.094238 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.094388 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:58.097097 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:58.097104 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:58.097108 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:58.097150 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:58.100928 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:58.101798 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:58.101803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:58.101807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:58.101982 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:58.102482 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:58.102488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:58.102492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:58.102942 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:58.103094 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:58.104470 (+ 1376us) server_negotiation.cc:299] Negotiation successful
0928 14:50:58.104703 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260928 14:50:58.105530 21506 tablet_copy_service.cc:140] P 9917d61e0e9d413b839dfc49f84766a4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d852e652ec6343c69c1e315766983c7b ({username='oryx', principal='oryx/127.14.241.252@KRBTEST.COM'} at 127.0.0.1:56290)
I20260928 14:50:58.105705 21506 tablet_copy_service.cc:161] P 9917d61e0e9d413b839dfc49f84766a4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d852e652ec6343c69c1e315766983c7b at {username='oryx', principal='oryx/127.14.241.252@KRBTEST.COM'} at 127.0.0.1:56290: session id = d852e652ec6343c69c1e315766983c7b-00000000000000000000000000000000
I20260928 14:50:58.109005 21506 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 14:50:58.110954 22053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 14:50:58.117986 22053 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: tablet copy: Starting download of 0 data blocks...
I20260928 14:50:58.118300 22053 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: tablet copy: Starting download of 1 WAL segments...
I20260928 14:50:58.121018 22053 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 14:50:58.125667 21506 tablet_copy_service.cc:342] P 9917d61e0e9d413b839dfc49f84766a4: Request end of tablet copy session d852e652ec6343c69c1e315766983c7b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.241.252@KRBTEST.COM'} at 127.0.0.1:56290
I20260928 14:50:58.125835 21506 tablet_copy_service.cc:433] P 9917d61e0e9d413b839dfc49f84766a4: ending tablet copy session d852e652ec6343c69c1e315766983c7b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d852e652ec6343c69c1e315766983c7b
I20260928 14:50:58.138788 22053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:58.141732 22254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:58.141911 22255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:58.146004 22053 server_base.cc:1034] running on GCE node
W20260928 14:50:58.146806 22257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:58.147324 22053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:58.147967 22053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:58.149153 22053 hybrid_clock.cc:648] HybridClock initialized: now 1790607058149124 us; error 23 us; skew 500 ppm
I20260928 14:50:58.149389 22053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:58 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607058, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:58.151463 22053 init.cc:377] Logged in from keytab as oryx/127.14.241.252@KRBTEST.COM (short username oryx)
I20260928 14:50:58.152740 22053 webserver.cc:460] Webserver started at http://127.14.241.252:42457/ using document root <none> and password file <none>
I20260928 14:50:58.153108 22053 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:58.153200 22053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:58.156353 22053 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260928 14:50:58.158286 22264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:58.158803 22053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:50:58.159015 22053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/wal
uuid: "d852e652ec6343c69c1e315766983c7b"
format_stamp: "Formatted at 2026-09-28 14:50:57 on dist-test-slave-vk40"
server_key: "f96f07605158e71307f6f97c5c86070d"
server_key_iv: "64f2548a288d269291e04fed4f1bec38"
server_key_version: "encryptionkey@0"
I20260928 14:50:58.159315 22053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:58.180661 22053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:58.184890 22053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:58.204736 22053 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.252:42347
I20260928 14:50:58.204790 22316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.252:42347 every 8 connection(s)
I20260928 14:50:58.206029 22053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
I20260928 14:50:58.211410 22317 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:50:58.215471 22317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Bootstrap starting.
I20260928 14:50:58.235690 22317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:50:58.236235 22317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Bootstrap complete.
I20260928 14:50:58.237372 22317 raft_consensus.cc:348] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } }
I20260928 14:50:58.237684 22317 raft_consensus.cc:729] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d852e652ec6343c69c1e315766983c7b, State: Initialized, Role: LEARNER
I20260928 14:50:58.237990 22317 consensus_queue.cc:260] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } }
I20260928 14:50:58.239159 22317 sys_catalog.cc:564] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:50:58.238867 22320 sys_catalog.cc:455] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
I20260928 14:50:58.239434 22320 sys_catalog.cc:458] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: This master's current role is: LEARNER
I20260928 14:50:58.247700 22331 catalog_manager.cc:1259] Loaded cluster ID: a0ac33f33ad647ed9c41bab5190b2b4d
I20260928 14:50:58.247826 22331 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: loading cluster ID for follower catalog manager: success
I20260928 14:50:58.251534 22331 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: acquiring CA information for follower catalog manager: success
I20260928 14:50:58.254055 22331 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:50:58.442108 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:58.435312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42347 (local address 127.0.0.1:45698)
0928 14:50:58.435503 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:58.435524 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:58.435673 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:58.436571 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:58.436576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:58.436605 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:58.436897 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:58.436909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.438187 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.438191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:58.439681 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:58.439697 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.441088 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.441094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:58.441352 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:58.441395 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 14:50:58.441480 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:50:58.441652 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260928 14:50:58.442260 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:58.435470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45698 (local address 127.14.241.252:42347)
0928 14:50:58.436182 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:58.436187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:58.436202 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:50:58.436260 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:58.436263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:58.436377 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:58.436484 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:58.437061 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.438024 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.439845 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.440872 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.441099 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:58.441305 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:58.441518 (+ 213us) server_negotiation.cc:299] Negotiation successful
0928 14:50:58.441696 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":457,"threads_started":1}
I20260928 14:50:58.443475 22292 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260928 14:50:58.444257 22320 sys_catalog.cc:455] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9917d61e0e9d413b839dfc49f84766a4. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
I20260928 14:50:58.444648 22320 sys_catalog.cc:458] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: This master's current role is: LEARNER
W20260928 14:50:58.496829 22253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:58.497145 22253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:58.497243 22253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:58.505266 22253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:58.505398 22253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:58.505511 22253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:50:58.515496 22253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab
--principal=oryx/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:44073
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=46501
--tserver_master_addrs=127.14.241.254:33413,127.14.241.253:35531,127.14.241.252:42347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:58.516837 22253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:58.518194 22253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:58.531132 22337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:58.531086 22340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:58.531155 22338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:58.532127 22253 server_base.cc:1034] running on GCE node
I20260928 14:50:58.532804 22253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:58.533795 22253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:58.535022 22253 hybrid_clock.cc:648] HybridClock initialized: now 1790607058534959 us; error 63 us; skew 500 ppm
I20260928 14:50:58.535364 22253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:58 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607058, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:58.538499 22253 init.cc:377] Logged in from keytab as oryx/127.14.241.193@KRBTEST.COM (short username oryx)
I20260928 14:50:58.539986 22253 webserver.cc:460] Webserver started at http://127.14.241.193:46501/ using document root <none> and password file <none>
I20260928 14:50:58.540815 22253 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:58.540961 22253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:58.546434 22253 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 14:50:58.549234 22347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:58.550586 22253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:50:58.550743 22253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "aaef801444514d3eb59008afc430a19a"
format_stamp: "Formatted at 2026-09-28 14:50:55 on dist-test-slave-vk40"
server_key: "2a6ce618ffa9bd6f34d4281cb68cab39"
server_key_iv: "baabaa0dd968f54759124e3420974afd"
server_key_version: "encryptionkey@0"
I20260928 14:50:58.551707 22253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:58.575431 22253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:58.580251 22253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:58.580616 22253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:58.581789 22253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:58.583272 22253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:58.583372 22253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:58.583478 22253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:58.583546 22253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:58.624792 22253 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:44073
I20260928 14:50:58.625056 22460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:44073 every 8 connection(s)
I20260928 14:50:58.626437 22253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:50:58.631129 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 22253
I20260928 14:50:58.631655 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 21682
Sep 28 14:50:58 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607058, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.253@KRBTEST.COM
Sep 28 14:50:58 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607058, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.252@KRBTEST.COM
I20260928 14:50:58.660594 22466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:58.629879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:59781 (local address 127.14.241.253:35531)
0928 14:50:58.636814 (+ 6935us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:58.636821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:58.637318 (+ 497us) server_negotiation.cc:407] Connection header received
0928 14:50:58.639617 (+ 2299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:58.639624 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:58.639841 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:58.639947 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:58.642543 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.643353 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.645528 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.646149 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.646257 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:58.652374 (+ 6117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:58.652382 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:58.652387 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:58.652429 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:58.656211 (+ 3782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:58.657731 (+ 1520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:58.657737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:58.657740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:58.657908 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:58.658486 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:58.658491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:58.658494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:58.658842 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:58.658978 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:58.659848 (+ 870us) server_negotiation.cc:299] Negotiation successful
0928 14:50:58.660067 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6790,"thread_start_us":6663,"threads_started":1}
I20260928 14:50:58.662549 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:58.630787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:35531 (local address 127.14.241.193:59781)
0928 14:50:58.637147 (+ 6360us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:58.637188 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:58.639272 (+ 2084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:58.640682 (+ 1410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:58.640692 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:58.641444 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:58.642313 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:58.642336 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.643554 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.643560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:58.645363 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:58.645378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.646636 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.646643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:58.646988 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:58.647805 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:58.647836 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:58.651010 (+ 3174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:58.656749 (+ 5739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:58.656759 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:58.656776 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:58.657585 (+ 809us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:58.658034 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:58.658038 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:58.658042 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:58.658346 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:58.658969 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:58.658977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:58.659483 (+ 506us) client_negotiation.cc:769] Sending connection context
0928 14:50:58.659724 (+ 241us) client_negotiation.cc:240] Negotiation successful
0928 14:50:58.660388 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6061,"thread_start_us":5846,"threads_started":1}
Sep 28 14:50:58 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.241.194@KRBTEST.COM
Sep 28 14:50:58 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607058, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:50:58.664897 22463 heartbeater.cc:347] Connected to a master server at 127.14.241.253:35531
I20260928 14:50:58.665263 22463 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:58.666268 22463 heartbeater.cc:511] Master 127.14.241.253:35531 requested a full tablet report, sending...
I20260928 14:50:58.668766 22089 ts_manager.cc:194] Registered new tserver with Master: aaef801444514d3eb59008afc430a19a (127.14.241.193:44073)
I20260928 14:50:58.668881 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:58.638343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:46025 (local address 127.14.241.252:42347)
0928 14:50:58.638542 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:58.638549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:58.650388 (+ 11839us) server_negotiation.cc:407] Connection header received
0928 14:50:58.650521 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:58.650527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:58.650614 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:58.650706 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:58.652109 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.653250 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.654961 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.655572 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.655664 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:58.662031 (+ 6367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:58.662038 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:58.662043 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:58.662086 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:58.665551 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:58.666525 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:58.666531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:58.666534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:58.666708 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:58.667160 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:58.667166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:58.667169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:58.667527 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:58.667647 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:58.668224 (+ 577us) server_negotiation.cc:299] Negotiation successful
0928 14:50:58.668477 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 14:50:58.669883 22471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:58.638100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42347 (local address 127.14.241.193:46025)
0928 14:50:58.650258 (+ 12158us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:58.650277 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:58.650401 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:58.650890 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:58.650895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:58.651530 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:58.651960 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:58.651973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.653415 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.653420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:58.654803 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:58.654816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.655993 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.655999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:58.656199 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:58.656756 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:58.656775 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:58.661715 (+ 4940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:58.665753 (+ 4038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:58.665758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:58.665762 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:58.666378 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:58.666874 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:58.666878 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:58.666882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:58.667033 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:58.667754 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:58.667759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:58.667928 (+ 169us) client_negotiation.cc:769] Sending connection context
0928 14:50:58.669121 (+ 1193us) client_negotiation.cc:240] Negotiation successful
0928 14:50:58.669429 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12024,"mutex_wait_us":3,"thread_start_us":11955,"threads_started":1}
I20260928 14:50:58.671293 22461 heartbeater.cc:347] Connected to a master server at 127.14.241.252:42347
I20260928 14:50:58.671446 22461 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:58.671988 22461 heartbeater.cc:511] Master 127.14.241.252:42347 requested a full tablet report, sending...
I20260928 14:50:58.672552 22470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:58.633167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59524 (local address 127.14.241.194:39459)
0928 14:50:58.644819 (+ 11652us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:58.644829 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:58.645549 (+ 720us) server_negotiation.cc:407] Connection header received
0928 14:50:58.645623 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:58.645632 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:58.645904 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:58.646052 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:58.648223 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.649745 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.653814 (+ 4069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.654696 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.654797 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:58.665095 (+ 10298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:58.665116 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:58.665135 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:58.665182 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:58.669008 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:58.669992 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:58.670002 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:58.670005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:58.670155 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:58.670625 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:58.670630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:58.670633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:58.671023 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:58.671261 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:58.671859 (+ 598us) server_negotiation.cc:299] Negotiation successful
0928 14:50:58.672127 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11467,"thread_start_us":11304,"threads_started":1}
I20260928 14:50:58.672601 22465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:58.629953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:56681 (local address 127.14.241.254:33413)
0928 14:50:58.650839 (+ 20886us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:58.650845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:58.650865 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:50:58.650923 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:58.650928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:58.651000 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:58.651113 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:58.656558 (+ 5445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.657644 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.659577 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.660325 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.660449 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:58.664926 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:58.664933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:58.664937 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:58.665034 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:58.668311 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:58.669913 (+ 1602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:58.669918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:58.669922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:58.670472 (+ 550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:58.671128 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:58.671133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:58.671135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:58.671461 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:58.671590 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:58.671995 (+ 405us) server_negotiation.cc:299] Negotiation successful
0928 14:50:58.672226 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20742,"thread_start_us":20642,"threads_started":1}
I20260928 14:50:58.673923 22281 ts_manager.cc:194] Registered new tserver with Master: aaef801444514d3eb59008afc430a19a (127.14.241.193:44073)
I20260928 14:50:58.674328 21806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:58.674393 22468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:58.631707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.193:56681)
0928 14:50:58.640105 (+ 8398us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:58.640124 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:58.640239 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:58.651127 (+ 10888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:58.651136 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:58.651624 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:58.652008 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:58.652021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.657809 (+ 5788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.657815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:58.659439 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:58.659451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:58.660527 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:58.660535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:58.660724 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:58.661332 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:58.661355 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:58.664713 (+ 3358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:58.669301 (+ 4588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:58.669306 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:58.669309 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:58.669744 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:58.670881 (+ 1137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:58.670886 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:58.670889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:58.671019 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:58.671568 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:58.671573 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:58.671752 (+ 179us) client_negotiation.cc:769] Sending connection context
0928 14:50:58.671874 (+ 122us) client_negotiation.cc:240] Negotiation successful
0928 14:50:58.672226 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8254,"mutex_wait_us":3,"thread_start_us":8027,"threads_started":1}
I20260928 14:50:58.675201 22462 heartbeater.cc:347] Connected to a master server at 127.14.241.254:33413
I20260928 14:50:58.675321 22462 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:58.675762 22462 heartbeater.cc:511] Master 127.14.241.254:33413 requested a full tablet report, sending...
I20260928 14:50:58.676898 21486 ts_manager.cc:194] Re-registered known tserver with Master: aaef801444514d3eb59008afc430a19a (127.14.241.193:44073)
I20260928 14:50:58.677953 21486 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:56681
I20260928 14:50:58.757448 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 21682
I20260928 14:50:58.770987 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:39459
--local_ip_for_outbound_sockets=127.14.241.194
--tserver_master_addrs=127.14.241.254:33413
--webserver_port=38145
--webserver_interface=127.14.241.194
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab
--principal=oryx/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.241.254:33413,127.14.241.253:35531
--tserver_master_addrs=127.14.241.254:33413,127.14.241.253:35531,127.14.241.252:42347 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:50:58.943003 22244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: NON_VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: true } } }
I20260928 14:50:58.943351 22244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:58.945518 22475 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4: attempting to promote NON_VOTER d852e652ec6343c69c1e315766983c7b to VOTER
I20260928 14:50:58.946199 22475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } }
I20260928 14:50:58.947985 22098 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 FOLLOWER]: Refusing update from remote peer 9917d61e0e9d413b839dfc49f84766a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260928 14:50:58.948094 22292 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 1 LEARNER]: Refusing update from remote peer 9917d61e0e9d413b839dfc49f84766a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260928 14:50:58.948637 22474 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260928 14:50:58.949095 22244 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260928 14:50:58.951416 22474 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d852e652ec6343c69c1e315766983c7b (127.14.241.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:50:58.952041 22098 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d852e652ec6343c69c1e315766983c7b (127.14.241.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:50:58.954989 22474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:50:58.955303 22474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:58.955227 22244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:50:58.955474 22244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:58.956911 22478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:50:58.957372 22478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:50:58.962188 22244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:50:58.962466 22244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:50:58.962914 22292 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d852e652ec6343c69c1e315766983c7b (127.14.241.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:50:58.966194 22479 sys_catalog.cc:455] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9917d61e0e9d413b839dfc49f84766a4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:50:58.966507 22479 sys_catalog.cc:458] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:50:59.092799 22473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:59.093102 22473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:59.093248 22473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:59.100486 22473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:59.100603 22473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:59.100749 22473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:50:59.110953 22473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab
--principal=oryx/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:39459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=38145
--tserver_master_addrs=127.14.241.254:33413,127.14.241.253:35531,127.14.241.252:42347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:59.112605 22473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:59.114285 22473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:59.129323 22484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:59.129410 22487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:59.129298 22485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:59.129968 22473 server_base.cc:1034] running on GCE node
I20260928 14:50:59.130586 22473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:59.131459 22473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:59.132639 22473 hybrid_clock.cc:648] HybridClock initialized: now 1790607059132583 us; error 51 us; skew 500 ppm
I20260928 14:50:59.132954 22473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:59 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607059, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:59.135937 22473 init.cc:377] Logged in from keytab as oryx/127.14.241.194@KRBTEST.COM (short username oryx)
I20260928 14:50:59.137357 22473 webserver.cc:460] Webserver started at http://127.14.241.194:38145/ using document root <none> and password file <none>
I20260928 14:50:59.138007 22473 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:59.138113 22473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:59.143540 22473 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260928 14:50:59.146207 22494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:59.147545 22473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260928 14:50:59.147698 22473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "c22174d815bd4fd29ffb608bef8e7786"
format_stamp: "Formatted at 2026-09-28 14:50:56 on dist-test-slave-vk40"
server_key: "fbd046596bf7af2ef5e4e78114c3aebc"
server_key_iv: "b77916a6859c379350a845a127a00e41"
server_key_version: "encryptionkey@0"
I20260928 14:50:59.148690 22473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:59.162443 22473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:59.166574 22473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:59.166834 22473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:59.167733 22473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:59.169106 22473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:59.169212 22473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:59.169314 22473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:59.169379 22473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:59.197929 22473 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:39459
I20260928 14:50:59.197973 22607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:39459 every 8 connection(s)
I20260928 14:50:59.199437 22473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:50:59.200032 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 22473
I20260928 14:50:59.200306 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 21816
Sep 28 14:50:59 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607059, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
Sep 28 14:50:59 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607059, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.252@KRBTEST.COM
Sep 28 14:50:59 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607059, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.253@KRBTEST.COM
Sep 28 14:50:59 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.241.195@KRBTEST.COM
I20260928 14:50:59.247568 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.203622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:33901 (local address 127.14.241.254:33413)
0928 14:50:59.212042 (+ 8420us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.212050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.217481 (+ 5431us) server_negotiation.cc:407] Connection header received
0928 14:50:59.220259 (+ 2778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.220266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.220396 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.220521 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:59.223293 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.224532 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.231058 (+ 6526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.231717 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.231832 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.238011 (+ 6179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:59.238016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:59.238019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:59.238067 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:59.241724 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.242919 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.242923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.242925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.243084 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.245267 (+ 2183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.245272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.245275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.245652 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:59.245799 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.246965 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.247197 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8285,"thread_start_us":8145,"threads_started":1}
I20260928 14:50:59.248620 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.202321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:55153 (local address 127.14.241.252:42347)
0928 14:50:59.215895 (+ 13574us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.215900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.216820 (+ 920us) server_negotiation.cc:407] Connection header received
0928 14:50:59.227289 (+ 10469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.227295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.227398 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.227497 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:59.230014 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.232945 (+ 2931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.235202 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.235842 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.235957 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.240578 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:59.240585 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:59.240590 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:59.240630 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:59.244458 (+ 3828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.245231 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.245237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.245241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.245400 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.245936 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.245942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.245945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.246284 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:59.246422 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.248002 (+ 1580us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.248219 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13448,"thread_start_us":13570,"threads_started":1}
I20260928 14:50:59.249010 22619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.217358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:46869 (local address 127.14.241.253:35531)
0928 14:50:59.232523 (+ 15165us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.232529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.232547 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:50:59.232600 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.232606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.232678 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.232775 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:59.234153 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.235243 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.237090 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.237679 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.237793 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.242080 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:59.242087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:59.242092 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:59.242147 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:59.245410 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.246155 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.246160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.246163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.246291 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.246720 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.246725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.246728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.247077 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:59.247224 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.248457 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.248673 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15046,"thread_start_us":16639,"threads_started":1}
I20260928 14:50:59.249200 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.202999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.194:33901)
0928 14:50:59.211341 (+ 8342us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:59.217402 (+ 6061us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:59.220099 (+ 2697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:59.221040 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:59.221055 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:59.222060 (+ 1005us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:59.223006 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.223037 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.224755 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.224761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.227080 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.227097 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.233370 (+ 6273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.233393 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.233714 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.234548 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:59.234582 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:59.237787 (+ 3205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:59.241872 (+ 4085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.241883 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.241904 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.242565 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.244394 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.244400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.244403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.244893 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.245954 (+ 1061us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:59.245967 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:59.246441 (+ 474us) client_negotiation.cc:769] Sending connection context
0928 14:50:59.247779 (+ 1338us) client_negotiation.cc:240] Negotiation successful
0928 14:50:59.248068 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7979,"thread_start_us":7726,"threads_started":1}
I20260928 14:50:59.249363 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.216958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:35531 (local address 127.14.241.194:46869)
0928 14:50:59.229828 (+ 12870us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:59.229846 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:59.229967 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:59.232940 (+ 2973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:59.232944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:59.233520 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:59.233888 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.233901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.235391 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.235396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.236906 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.236926 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.238027 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.238033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.238227 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.238731 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:59.238752 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:59.241903 (+ 3151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:59.245533 (+ 3630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.245538 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.245542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.246032 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.246408 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.246412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.246416 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.246574 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.247280 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:59.247284 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:59.247417 (+ 133us) client_negotiation.cc:769] Sending connection context
0928 14:50:59.247535 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 14:50:59.247897 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12735,"thread_start_us":13070,"threads_started":1}
I20260928 14:50:59.250012 22617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.208655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42347 (local address 127.14.241.194:55153)
0928 14:50:59.216646 (+ 7991us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:59.216691 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:59.219690 (+ 2999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:59.228958 (+ 9268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:59.228964 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:59.229495 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:59.229803 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.229819 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.233101 (+ 3282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.233105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.235037 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.235052 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.236208 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.236227 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.236426 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.237051 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:59.237072 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:59.240348 (+ 3276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:59.244605 (+ 4257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.244610 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.244614 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.245096 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.245533 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.245538 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.245542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.245803 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.246489 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:59.246493 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:59.246647 (+ 154us) client_negotiation.cc:769] Sending connection context
0928 14:50:59.246817 (+ 170us) client_negotiation.cc:240] Negotiation successful
0928 14:50:59.247363 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7885,"spinlock_wait_cycles":25984,"thread_start_us":7759,"threads_started":1}
I20260928 14:50:59.250757 22610 heartbeater.cc:347] Connected to a master server at 127.14.241.253:35531
I20260928 14:50:59.250975 22609 heartbeater.cc:347] Connected to a master server at 127.14.241.254:33413
I20260928 14:50:59.251084 22610 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:59.251112 22609 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:59.251559 22608 heartbeater.cc:347] Connected to a master server at 127.14.241.252:42347
I20260928 14:50:59.251686 22608 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:59.252132 22610 heartbeater.cc:511] Master 127.14.241.253:35531 requested a full tablet report, sending...
I20260928 14:50:59.252154 22608 heartbeater.cc:511] Master 127.14.241.252:42347 requested a full tablet report, sending...
I20260928 14:50:59.252504 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.201599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55526 (local address 127.14.241.195:42739)
0928 14:50:59.228150 (+ 26551us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.228160 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.230218 (+ 2058us) server_negotiation.cc:407] Connection header received
0928 14:50:59.230294 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.230302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.230579 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.230721 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:59.233995 (+ 3274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.235415 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.237251 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.238941 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.239057 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.244573 (+ 5516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:59.244588 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:59.244607 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:59.244654 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:59.248171 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.249019 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.249027 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.249030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.249193 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.250062 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.250067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.250069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.250937 (+ 868us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:59.251240 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.251818 (+ 578us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.252096 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26353,"thread_start_us":26184,"threads_started":1}
I20260928 14:50:59.252730 22609 heartbeater.cc:511] Master 127.14.241.254:33413 requested a full tablet report, sending...
I20260928 14:50:59.253836 21940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:50:59.253860 21486 ts_manager.cc:194] Re-registered known tserver with Master: c22174d815bd4fd29ffb608bef8e7786 (127.14.241.194:39459)
I20260928 14:50:59.254153 22281 ts_manager.cc:194] Registered new tserver with Master: c22174d815bd4fd29ffb608bef8e7786 (127.14.241.194:39459)
I20260928 14:50:59.254264 22089 ts_manager.cc:194] Registered new tserver with Master: c22174d815bd4fd29ffb608bef8e7786 (127.14.241.194:39459)
I20260928 14:50:59.255010 21486 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:33901
I20260928 14:50:59.337163 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 21816
I20260928 14:50:59.349746 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:42739
--local_ip_for_outbound_sockets=127.14.241.195
--tserver_master_addrs=127.14.241.254:33413
--webserver_port=44793
--webserver_interface=127.14.241.195
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab
--principal=oryx/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.241.254:33413,127.14.241.253:35531
--tserver_master_addrs=127.14.241.254:33413,127.14.241.253:35531,127.14.241.252:42347 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:50:59.662078 22621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:50:59.662340 22621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:50:59.662439 22621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:50:59.669464 22621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:50:59.669571 22621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:50:59.669687 22621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:50:59.679742 22621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab
--principal=oryx/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:42739
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=44793
--tserver_master_addrs=127.14.241.254:33413,127.14.241.253:35531,127.14.241.252:42347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:50:59.680231 22462 heartbeater.cc:503] Master 127.14.241.254:33413 was elected leader, sending a full tablet report...
I20260928 14:50:59.681290 22621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:50:59.682667 22621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:50:59.695598 22626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:59.695677 22627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:50:59.695633 22629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:50:59.697633 22621 server_base.cc:1034] running on GCE node
I20260928 14:50:59.698280 22621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:50:59.699252 22621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:50:59.700449 22621 hybrid_clock.cc:648] HybridClock initialized: now 1790607059700397 us; error 46 us; skew 500 ppm
I20260928 14:50:59.700763 22621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:50:59 dist-test-slave-vk40 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607059, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:50:59.703748 22621 init.cc:377] Logged in from keytab as oryx/127.14.241.195@KRBTEST.COM (short username oryx)
I20260928 14:50:59.705343 22621 webserver.cc:460] Webserver started at http://127.14.241.195:44793/ using document root <none> and password file <none>
I20260928 14:50:59.706043 22621 fs_manager.cc:359] Metadata directory not provided
I20260928 14:50:59.706167 22621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:50:59.711496 22621 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 14:50:59.714069 22636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:59.715365 22621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:50:59.715510 22621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "b0fc3d05546d4301baae6c6202709f82"
format_stamp: "Formatted at 2026-09-28 14:50:56 on dist-test-slave-vk40"
server_key: "db9f633cd38cb1d00d300ab6052fcd1b"
server_key_iv: "18647a7371710153a424b4caab467f22"
server_key_version: "encryptionkey@0"
I20260928 14:50:59.716490 22621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:50:59.735919 22621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:50:59.740304 22621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:50:59.740571 22621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:50:59.741564 22621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:50:59.742918 22621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:50:59.743010 22621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:59.743109 22621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:50:59.743176 22621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:50:59.771704 22621 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:42739
I20260928 14:50:59.771756 22748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:42739 every 8 connection(s)
I20260928 14:50:59.773136 22621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:50:59.778503 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 22621
Sep 28 14:50:59 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607059, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
Sep 28 14:50:59 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607059, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.252@KRBTEST.COM
Sep 28 14:50:59 dist-test-slave-vk40 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607059, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.253@KRBTEST.COM
I20260928 14:50:59.833225 22755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.775856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:39563 (local address 127.14.241.254:33413)
0928 14:50:59.806296 (+ 30440us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.806302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.806327 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:50:59.806405 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.806412 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.806498 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.806629 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:59.809514 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.810636 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.813355 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.814008 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.814124 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.819965 (+ 5841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:59.819972 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:59.819978 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:59.820024 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:59.823555 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.829526 (+ 5971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.829532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.829535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.829698 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.830246 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.830250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.830253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.830643 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:59.830773 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.832593 (+ 1820us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.832844 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":30297,"thread_start_us":30175,"threads_started":1}
I20260928 14:50:59.835489 22756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.777078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42347 (local address 127.14.241.195:35507)
0928 14:50:59.798499 (+ 21421us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:59.798517 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:59.801092 (+ 2575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:59.807552 (+ 6460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:59.807558 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:59.808164 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:59.809274 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.809303 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.810941 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.810946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.814539 (+ 3593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.814551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.818370 (+ 3819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.818377 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.818579 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.819951 (+ 1372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:59.819973 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:59.824347 (+ 4374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:59.828089 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.828099 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.828120 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.828797 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.831972 (+ 3175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.831976 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.831979 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.832163 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.833825 (+ 1662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:59.833830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:59.833993 (+ 163us) client_negotiation.cc:769] Sending connection context
0928 14:50:59.834122 (+ 129us) client_negotiation.cc:240] Negotiation successful
0928 14:50:59.834391 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21289,"mutex_wait_us":2022,"spinlock_wait_cycles":3840,"thread_start_us":19448,"threads_started":1}
I20260928 14:50:59.836154 22757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.776669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.195:39563)
0928 14:50:59.797067 (+ 20398us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:59.797134 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:59.800665 (+ 3531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:59.807017 (+ 6352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:59.807031 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:59.808048 (+ 1017us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:59.809275 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.809302 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.810795 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.810802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.813189 (+ 2387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.813207 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.814801 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.814808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.815119 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.816506 (+ 1387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:59.816542 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:59.819687 (+ 3145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:59.828941 (+ 9254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.828946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.828950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.829380 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.829845 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.829849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.829852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.830129 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.830872 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:59.830892 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:59.831349 (+ 457us) client_negotiation.cc:769] Sending connection context
0928 14:50:59.832490 (+ 1141us) client_negotiation.cc:240] Negotiation successful
0928 14:50:59.833941 (+ 1451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19977,"mutex_wait_us":53,"spinlock_wait_cycles":21760,"thread_start_us":21363,"threads_started":1}
I20260928 14:50:59.837291 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.775791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:35507 (local address 127.14.241.252:42347)
0928 14:50:59.807180 (+ 31389us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.807186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.807206 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:50:59.807267 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.807273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.807359 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.807868 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:59.809667 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.810775 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.815549 (+ 4774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.816220 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.818135 (+ 1915us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.824599 (+ 6464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:59.824606 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:59.824612 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:59.824657 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:59.827920 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.831705 (+ 3785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.831710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.831714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.831851 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.832309 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.832314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.832317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.833625 (+ 1308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:59.833765 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.836688 (+ 2923us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.836931 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31265,"thread_start_us":31174,"threads_started":1}
I20260928 14:50:59.837671 22750 heartbeater.cc:347] Connected to a master server at 127.14.241.254:33413
I20260928 14:50:59.838009 22750 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:59.838016 22749 heartbeater.cc:347] Connected to a master server at 127.14.241.252:42347
I20260928 14:50:59.838200 22749 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:59.839128 22750 heartbeater.cc:511] Master 127.14.241.254:33413 requested a full tablet report, sending...
I20260928 14:50:59.839151 22749 heartbeater.cc:511] Master 127.14.241.252:42347 requested a full tablet report, sending...
I20260928 14:50:59.839749 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.797948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:41877 (local address 127.14.241.253:35531)
0928 14:50:59.804667 (+ 6719us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.804674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.806824 (+ 2150us) server_negotiation.cc:407] Connection header received
0928 14:50:59.806981 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.806986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.807080 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.807175 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:59.814196 (+ 7021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.815388 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.819682 (+ 4294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.820328 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.820460 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.827281 (+ 6821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:59.827288 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:59.827294 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:59.827334 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:59.830838 (+ 3504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.835443 (+ 4605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.835453 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.835456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.835642 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.836118 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.836123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.836126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.837387 (+ 1261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:59.837797 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.839077 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.839320 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6560,"thread_start_us":6448,"threads_started":1}
I20260928 14:50:59.840197 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.797830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:35531 (local address 127.14.241.195:41877)
0928 14:50:59.806695 (+ 8865us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:59.806711 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:59.806832 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:59.807349 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:59.807353 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:59.808813 (+ 1460us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:50:59.809285 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.809302 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.816196 (+ 6894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.816201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.819492 (+ 3291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.819508 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.821281 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.821288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.821430 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.822157 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:50:59.822179 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:50:59.826619 (+ 4440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:50:59.831963 (+ 5344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.831968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.831971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.833156 (+ 1185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.835790 (+ 2634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:50:59.835794 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:50:59.835798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:50:59.835975 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:50:59.838041 (+ 2066us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:50:59.838046 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:50:59.838213 (+ 167us) client_negotiation.cc:769] Sending connection context
0928 14:50:59.838358 (+ 145us) client_negotiation.cc:240] Negotiation successful
0928 14:50:59.838664 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8729,"mutex_wait_us":196,"thread_start_us":8621,"threads_started":1}
I20260928 14:50:59.840826 21486 ts_manager.cc:194] Re-registered known tserver with Master: b0fc3d05546d4301baae6c6202709f82 (127.14.241.195:42739)
I20260928 14:50:59.841284 22281 ts_manager.cc:194] Registered new tserver with Master: b0fc3d05546d4301baae6c6202709f82 (127.14.241.195:42739)
I20260928 14:50:59.841836 21486 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:39563
I20260928 14:50:59.843286 22751 heartbeater.cc:347] Connected to a master server at 127.14.241.253:35531
I20260928 14:50:59.843415 22751 heartbeater.cc:464] Registering TS with master...
I20260928 14:50:59.843911 22751 heartbeater.cc:511] Master 127.14.241.253:35531 requested a full tablet report, sending...
I20260928 14:50:59.845008 22089 ts_manager.cc:194] Registered new tserver with Master: b0fc3d05546d4301baae6c6202709f82 (127.14.241.195:42739)
I20260928 14:50:59.846998 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.809881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42362 (local address 127.14.241.253:35531)
0928 14:50:59.824987 (+ 15106us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.824993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.825013 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:50:59.825069 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.825075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.825152 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.825260 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:59.827474 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.828593 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.832803 (+ 4210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.833301 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.833433 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.839762 (+ 6329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:59.839770 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:59.839775 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:59.839811 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:59.843826 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.844840 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.844846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.844850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.845024 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.845570 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.845577 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.845580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.845865 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:59.846201 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.846461 (+ 260us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.846624 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14952,"thread_start_us":14820,"threads_started":1}
I20260928 14:50:59.855831 22766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.811555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56302 (local address 127.14.241.254:33413)
0928 14:50:59.825137 (+ 13582us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.825143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.826760 (+ 1617us) server_negotiation.cc:407] Connection header received
0928 14:50:59.826821 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.826825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.826898 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.827000 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:59.830229 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.831824 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.835890 (+ 4066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.836782 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.836889 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.850426 (+ 13537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:59.850431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:59.850434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:59.850471 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:59.853259 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.853957 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.853961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.853963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.854093 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.854451 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.854454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.854456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.854741 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:59.854987 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.855284 (+ 297us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.855480 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":382,"server-negotiator.queue_time_us":13454,"thread_start_us":14146,"threads_started":1}
I20260928 14:50:59.858970 22767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.811805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.14.241.252:42347)
0928 14:50:59.830803 (+ 18998us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.830809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.833265 (+ 2456us) server_negotiation.cc:407] Connection header received
0928 14:50:59.833592 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.833596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.833700 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.833782 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:50:59.847619 (+ 13837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.848959 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.850254 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.850718 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.850824 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.853107 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:50:59.853114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:50:59.853119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:50:59.853158 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:50:59.856255 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.856828 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.856832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.856834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.857066 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:50:59.857383 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:50:59.857387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:50:59.857389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:50:59.857682 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:50:59.857996 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.858286 (+ 290us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.858523 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18876,"thread_start_us":18746,"threads_started":1}
I20260928 14:50:59.859812 21486 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:50:59.867028 21486 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:50:59.897035 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.886105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:44073 (local address 127.0.0.1:34752)
0928 14:50:59.886604 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:59.886621 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:59.886845 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:59.888110 (+ 1265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:59.888115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:59.888143 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:59.888721 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.888735 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.891345 (+ 2610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.891352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.893783 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.893811 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.896115 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.896122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.896353 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.896392 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 14:50:59.896483 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 14:50:59.896630 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":278,"threads_started":1}
I20260928 14:50:59.897859 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.886410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34752 (local address 127.14.241.193:44073)
0928 14:50:59.887170 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.887176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.887199 (+ 23us) server_negotiation.cc:407] Connection header received
0928 14:50:59.887285 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.887290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.887576 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.887818 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:59.888979 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.891127 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.893996 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.895558 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.895729 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.895949 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.897329 (+ 1380us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.897519 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":513,"threads_started":1}
I20260928 14:50:59.902216 22395 tablet_service.cc:1467] Processing CreateTablet for tablet e4b1b3f0ce4e4e9291483d1d793541e9 (DEFAULT_TABLE table=test-table [id=9bd91dc5f15742ba9cbf19b10de8a53c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:59.903555 22395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b1b3f0ce4e4e9291483d1d793541e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:59.906263 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.884871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:42739 (local address 127.0.0.1:55532)
0928 14:50:59.885519 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:59.885535 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:59.885630 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:59.899671 (+ 14041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:59.899676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:59.899702 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:59.900073 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.900081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.902237 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.902241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.903812 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.903825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.905639 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.905644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.905774 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.905799 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 14:50:59.905894 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 14:50:59.906010 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":380,"threads_started":1}
I20260928 14:50:59.906800 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.885173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55532 (local address 127.14.241.195:42739)
0928 14:50:59.899049 (+ 13876us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.899059 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.899092 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:50:59.899194 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.899203 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.899496 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.899687 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:59.900249 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.902110 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.903950 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.905437 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.905629 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.905804 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.905919 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.906080 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13681,"thread_start_us":14727,"threads_started":1}
I20260928 14:50:59.910247 22684 tablet_service.cc:1467] Processing CreateTablet for tablet e4b1b3f0ce4e4e9291483d1d793541e9 (DEFAULT_TABLE table=test-table [id=9bd91dc5f15742ba9cbf19b10de8a53c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:59.911859 22684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b1b3f0ce4e4e9291483d1d793541e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:59.917620 22784 tablet_bootstrap.cc:492] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: Bootstrap starting.
I20260928 14:50:59.920133 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.887559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:39459 (local address 127.0.0.1:59540)
0928 14:50:59.899267 (+ 11708us) negotiation.cc:107] Waiting for socket to connect
0928 14:50:59.899288 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:50:59.899393 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:50:59.913544 (+ 14151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:50:59.913550 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:50:59.913586 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:50:59.913779 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.913790 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.915671 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.915677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.917602 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:50:59.917617 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.919329 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.919335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:50:59.919517 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.919552 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:50:59.919656 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 14:50:59.919805 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11598,"thread_start_us":12619,"threads_started":1}
I20260928 14:50:59.920187 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:50:59.887796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59540 (local address 127.14.241.194:39459)
0928 14:50:59.912858 (+ 25062us) server_negotiation.cc:206] Beginning negotiation
0928 14:50:59.912867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:50:59.912913 (+ 46us) server_negotiation.cc:407] Connection header received
0928 14:50:59.913016 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:50:59.913024 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:50:59.913284 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:50:59.913515 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:50:59.913978 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.915522 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.917796 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:50:59.919154 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:50:59.919328 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:50:59.919498 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0928 14:50:59.919753 (+ 255us) server_negotiation.cc:299] Negotiation successful
0928 14:50:59.919922 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24870,"thread_start_us":25864,"threads_started":1}
I20260928 14:50:59.921440 22784 tablet_bootstrap.cc:654] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:59.922675 22784 log.cc:824] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:59.924064 22542 tablet_service.cc:1467] Processing CreateTablet for tablet e4b1b3f0ce4e4e9291483d1d793541e9 (DEFAULT_TABLE table=test-table [id=9bd91dc5f15742ba9cbf19b10de8a53c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:50:59.925999 22542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b1b3f0ce4e4e9291483d1d793541e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:50:59.926679 22786 tablet_bootstrap.cc:492] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: Bootstrap starting.
I20260928 14:50:59.929585 22786 tablet_bootstrap.cc:654] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:59.930763 22786 log.cc:824] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:59.932663 22784 tablet_bootstrap.cc:492] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: No bootstrap required, opened a new log
I20260928 14:50:59.932964 22784 ts_tablet_manager.cc:1397] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.006s
I20260928 14:50:59.933176 22786 tablet_bootstrap.cc:492] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: No bootstrap required, opened a new log
I20260928 14:50:59.933396 22786 ts_tablet_manager.cc:1397] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260928 14:50:59.941875 22786 raft_consensus.cc:348] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } } peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } }
I20260928 14:50:59.942302 22786 raft_consensus.cc:374] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:59.942392 22786 raft_consensus.cc:729] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fc3d05546d4301baae6c6202709f82, State: Initialized, Role: FOLLOWER
I20260928 14:50:59.943010 22788 tablet_bootstrap.cc:492] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: Bootstrap starting.
I20260928 14:50:59.943188 22784 raft_consensus.cc:348] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } } peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } }
I20260928 14:50:59.943524 22784 raft_consensus.cc:374] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:59.943624 22784 raft_consensus.cc:729] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaef801444514d3eb59008afc430a19a, State: Initialized, Role: FOLLOWER
I20260928 14:50:59.943714 22786 consensus_queue.cc:260] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } } peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } }
I20260928 14:50:59.944164 22784 consensus_queue.cc:260] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } } peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } }
I20260928 14:50:59.945494 22786 ts_tablet_manager.cc:1428] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
I20260928 14:50:59.945951 22750 heartbeater.cc:503] Master 127.14.241.254:33413 was elected leader, sending a full tablet report...
I20260928 14:50:59.946970 22788 tablet_bootstrap.cc:654] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: Neither blocks nor log segments found. Creating new log.
I20260928 14:50:59.952325 22784 ts_tablet_manager.cc:1428] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: Time spent starting tablet: real 0.019s user 0.017s sys 0.003s
I20260928 14:50:59.948236 22788 log.cc:824] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: Log is configured to *not* fsync() on all Append() calls
I20260928 14:50:59.962071 22788 tablet_bootstrap.cc:492] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: No bootstrap required, opened a new log
I20260928 14:50:59.962320 22788 ts_tablet_manager.cc:1397] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.000s
I20260928 14:50:59.969810 22788 raft_consensus.cc:348] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } } peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } }
I20260928 14:50:59.970163 22788 raft_consensus.cc:374] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:50:59.970240 22788 raft_consensus.cc:729] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22174d815bd4fd29ffb608bef8e7786, State: Initialized, Role: FOLLOWER
I20260928 14:50:59.970765 22788 consensus_queue.cc:260] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } } peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } }
I20260928 14:50:59.971745 22609 heartbeater.cc:503] Master 127.14.241.254:33413 was elected leader, sending a full tablet report...
I20260928 14:50:59.972388 22788 ts_tablet_manager.cc:1428] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: Time spent starting tablet: real 0.010s user 0.012s sys 0.000s
W20260928 14:51:00.043741 22752 tablet.cc:2369] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:51:00.137184 22464 tablet.cc:2369] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:51:00.179504 22789 raft_consensus.cc:482] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:51:00.179965 22789 raft_consensus.cc:504] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } } peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } }
I20260928 14:51:00.181810 22789 leader_election.cc:288] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaef801444514d3eb59008afc430a19a (127.14.241.193:44073), c22174d815bd4fd29ffb608bef8e7786 (127.14.241.194:39459)
I20260928 14:51:00.187513 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:00.182269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:60075 (local address 127.14.241.193:44073)
0928 14:51:00.182456 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:00.182462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:00.182485 (+ 23us) server_negotiation.cc:407] Connection header received
0928 14:51:00.182638 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:00.182644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:00.182711 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:00.182783 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:00.183197 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.184011 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.185759 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.186598 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.186725 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:00.186778 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:00.187143 (+ 365us) server_negotiation.cc:299] Negotiation successful
0928 14:51:00.187268 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:51:00.187547 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:00.182034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:44073 (local address 127.14.241.195:60075)
0928 14:51:00.182326 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:00.182345 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:00.182461 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:00.182824 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:00.182828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:00.182865 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:00.183045 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:00.183056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.184151 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.184156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:00.185599 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:00.185611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.186774 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.186780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:00.186980 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:00.187010 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 14:51:00.187094 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:51:00.187199 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":19,"spinlock_wait_cycles":1792}
I20260928 14:51:00.187597 22757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:00.182053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:39459 (local address 127.14.241.195:42491)
0928 14:51:00.182735 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:00.182752 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:00.182854 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:00.183197 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:00.183201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:00.183225 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:00.183393 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:00.183403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.184533 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.184536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:00.185871 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:00.185883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.186938 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.186942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:00.187098 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:00.187130 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 14:51:00.187215 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:51:00.187327 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":455}
I20260928 14:51:00.187599 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:00.182416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:42491 (local address 127.14.241.194:39459)
0928 14:51:00.182620 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:00.182625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:00.182930 (+ 305us) server_negotiation.cc:407] Connection header received
0928 14:51:00.183007 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:00.183012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:00.183082 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:00.183169 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:00.183515 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.184401 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.186014 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.186803 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.186950 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:00.187018 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:00.187251 (+ 233us) server_negotiation.cc:299] Negotiation successful
0928 14:51:00.187380 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 14:51:00.188519 22563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b1b3f0ce4e4e9291483d1d793541e9" candidate_uuid: "b0fc3d05546d4301baae6c6202709f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22174d815bd4fd29ffb608bef8e7786" is_pre_election: true
I20260928 14:51:00.188534 22415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b1b3f0ce4e4e9291483d1d793541e9" candidate_uuid: "b0fc3d05546d4301baae6c6202709f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaef801444514d3eb59008afc430a19a" is_pre_election: true
I20260928 14:51:00.188975 22563 raft_consensus.cc:2454] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0fc3d05546d4301baae6c6202709f82 in term 0.
I20260928 14:51:00.188975 22415 raft_consensus.cc:2454] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0fc3d05546d4301baae6c6202709f82 in term 0.
I20260928 14:51:00.189587 22639 leader_election.cc:302] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0fc3d05546d4301baae6c6202709f82, c22174d815bd4fd29ffb608bef8e7786; no voters:
I20260928 14:51:00.189936 22789 raft_consensus.cc:2781] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:51:00.190083 22789 raft_consensus.cc:482] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:51:00.190181 22789 raft_consensus.cc:3037] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:00.192217 22789 raft_consensus.cc:504] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } } peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } }
I20260928 14:51:00.192874 22789 leader_election.cc:288] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [CANDIDATE]: Term 1 election: Requested vote from peers aaef801444514d3eb59008afc430a19a (127.14.241.193:44073), c22174d815bd4fd29ffb608bef8e7786 (127.14.241.194:39459)
I20260928 14:51:00.193264 22415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b1b3f0ce4e4e9291483d1d793541e9" candidate_uuid: "b0fc3d05546d4301baae6c6202709f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaef801444514d3eb59008afc430a19a"
I20260928 14:51:00.193303 22563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b1b3f0ce4e4e9291483d1d793541e9" candidate_uuid: "b0fc3d05546d4301baae6c6202709f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22174d815bd4fd29ffb608bef8e7786"
I20260928 14:51:00.193454 22415 raft_consensus.cc:3037] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:00.193472 22563 raft_consensus.cc:3037] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:00.195617 22563 raft_consensus.cc:2454] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fc3d05546d4301baae6c6202709f82 in term 1.
I20260928 14:51:00.195806 22415 raft_consensus.cc:2454] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0fc3d05546d4301baae6c6202709f82 in term 1.
I20260928 14:51:00.196024 22639 leader_election.cc:302] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0fc3d05546d4301baae6c6202709f82, c22174d815bd4fd29ffb608bef8e7786; no voters:
I20260928 14:51:00.196302 22789 raft_consensus.cc:2781] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:00.196758 22789 raft_consensus.cc:686] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 1 LEADER]: Becoming Leader. State: Replica: b0fc3d05546d4301baae6c6202709f82, State: Running, Role: LEADER
I20260928 14:51:00.197291 22789 consensus_queue.cc:237] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } } peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } }
I20260928 14:51:00.203109 21486 catalog_manager.cc:5495] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0fc3d05546d4301baae6c6202709f82 (127.14.241.195). New cstate: current_term: 1 leader_uuid: "b0fc3d05546d4301baae6c6202709f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } health_report { overall_health: UNKNOWN } } }
W20260928 14:51:00.214730 22616 tablet.cc:2369] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:51:00.265009 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:00.260267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55542 (local address 127.14.241.195:42739)
0928 14:51:00.260436 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:00.260442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:00.260590 (+ 148us) server_negotiation.cc:407] Connection header received
0928 14:51:00.260800 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:00.260806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:00.260947 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:00.261086 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:51:00.261367 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.262031 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.263206 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.263595 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.263676 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:00.264038 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:51:00.264579 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:51:00.264688 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:00.264724 (+ 36us) server_negotiation.cc:299] Negotiation successful
0928 14:51:00.264790 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:51:00.273042 22563 raft_consensus.cc:1261] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786 [term 1 FOLLOWER]: Refusing update from remote peer b0fc3d05546d4301baae6c6202709f82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:00.273042 22415 raft_consensus.cc:1261] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a [term 1 FOLLOWER]: Refusing update from remote peer b0fc3d05546d4301baae6c6202709f82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:00.274132 22793 consensus_queue.cc:1035] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:00.274493 22789 consensus_queue.cc:1035] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:00.285737 22796 mvcc.cc:205] Tried to move back new op lower bound from 7334326518867529728 to 7334326518572343296. Current Snapshot: MvccSnapshot[applied={T|T < 7334326518867529728}]
I20260928 14:51:00.288807 22797 mvcc.cc:205] Tried to move back new op lower bound from 7334326518867529728 to 7334326518572343296. Current Snapshot: MvccSnapshot[applied={T|T < 7334326518867529728}]
I20260928 14:51:00.309417 21486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:00.309645 21486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:00.316012 21486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9917d61e0e9d413b839dfc49f84766a4: Sending DeleteTablet for 3 replicas of tablet e4b1b3f0ce4e4e9291483d1d793541e9
I20260928 14:51:00.317029 22684 tablet_service.cc:1514] Processing DeleteTablet for tablet e4b1b3f0ce4e4e9291483d1d793541e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:00 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:55532
I20260928 14:51:00.318140 22395 tablet_service.cc:1514] Processing DeleteTablet for tablet e4b1b3f0ce4e4e9291483d1d793541e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:00 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:34752
I20260928 14:51:00.318976 22806 tablet_replica.cc:330] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: stopping tablet replica
I20260928 14:51:00.319367 22806 raft_consensus.cc:2229] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:00.319852 22806 raft_consensus.cc:2258] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:00.321683 22806 ts_tablet_manager.cc:1905] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:00.323881 22542 tablet_service.cc:1514] Processing DeleteTablet for tablet e4b1b3f0ce4e4e9291483d1d793541e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:00 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59540
I20260928 14:51:00.329438 22805 tablet_replica.cc:330] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: stopping tablet replica
I20260928 14:51:00.330075 22805 raft_consensus.cc:2229] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:51:00.330962 22805 raft_consensus.cc:2258] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:00.333060 22805 ts_tablet_manager.cc:1905] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:00.335585 22806 ts_tablet_manager.cc:1918] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:00.335839 22806 log.cc:1196] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/e4b1b3f0ce4e4e9291483d1d793541e9
I20260928 14:51:00.336932 22806 ts_tablet_manager.cc:1939] T e4b1b3f0ce4e4e9291483d1d793541e9 P aaef801444514d3eb59008afc430a19a: Deleting consensus metadata
I20260928 14:51:00.339184 21470 catalog_manager.cc:4841] TS aaef801444514d3eb59008afc430a19a (127.14.241.193:44073): tablet e4b1b3f0ce4e4e9291483d1d793541e9 (table test-table [id=9bd91dc5f15742ba9cbf19b10de8a53c]) successfully deleted
I20260928 14:51:00.340822 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 21453
I20260928 14:51:00.341830 21516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:00.346210 22805 ts_tablet_manager.cc:1918] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:00.353636 22805 log.cc:1196] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/e4b1b3f0ce4e4e9291483d1d793541e9
I20260928 14:51:00.354241 22805 ts_tablet_manager.cc:1939] T e4b1b3f0ce4e4e9291483d1d793541e9 P b0fc3d05546d4301baae6c6202709f82: Deleting consensus metadata
I20260928 14:51:00.355520 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:00.327530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42372 (local address 127.14.241.253:35531)
0928 14:51:00.327720 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:00.327726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:00.336900 (+ 9174us) server_negotiation.cc:407] Connection header received
0928 14:51:00.337214 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:00.337221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:00.337319 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:00.337407 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:00.339835 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.340954 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.343433 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.344115 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.344213 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:00.348088 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:00.348096 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:00.348102 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:00.348146 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:00.351768 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:00.352751 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:00.352758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:00.352762 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:00.352948 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:00.353463 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:00.353468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:00.353473 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:00.353846 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:00.354104 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:00.354907 (+ 803us) server_negotiation.cc:299] Negotiation successful
0928 14:51:00.355159 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:51:00.355753 22807 tablet_replica.cc:330] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: stopping tablet replica
I20260928 14:51:00.356498 22807 raft_consensus.cc:2229] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:00.357033 22807 raft_consensus.cc:2258] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:00.360975 22807 ts_tablet_manager.cc:1905] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:00.368305 22807 ts_tablet_manager.cc:1918] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:00.368593 22807 log.cc:1196] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/e4b1b3f0ce4e4e9291483d1d793541e9
I20260928 14:51:00.369189 22807 ts_tablet_manager.cc:1939] T e4b1b3f0ce4e4e9291483d1d793541e9 P c22174d815bd4fd29ffb608bef8e7786: Deleting consensus metadata
I20260928 14:51:00.373504 22767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:00.340066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.14.241.252:42347)
0928 14:51:00.340268 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:00.340274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:00.356936 (+ 16662us) server_negotiation.cc:407] Connection header received
0928 14:51:00.357238 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:00.357244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:00.357322 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:00.357389 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:00.359200 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.360362 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.362233 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.362875 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.362977 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:00.366457 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:00.366464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:00.366468 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:00.366513 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:00.370013 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:00.371073 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:00.371079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:00.371083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:00.371238 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:00.371686 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:00.371691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:00.371694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:00.372083 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:00.372320 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:00.372919 (+ 599us) server_negotiation.cc:299] Negotiation successful
0928 14:51:00.373151 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260928 14:51:00.482226 21470 catalog_manager.cc:4841] TS b0fc3d05546d4301baae6c6202709f82 (127.14.241.195:42739): tablet e4b1b3f0ce4e4e9291483d1d793541e9 (table test-table [id=9bd91dc5f15742ba9cbf19b10de8a53c]) successfully deleted
I20260928 14:51:00.484977 21471 catalog_manager.cc:4841] TS c22174d815bd4fd29ffb608bef8e7786 (127.14.241.194:39459): tablet e4b1b3f0ce4e4e9291483d1d793541e9 (table test-table [id=9bd91dc5f15742ba9cbf19b10de8a53c]) successfully deleted
I20260928 14:51:00.485599 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 21453
I20260928 14:51:00.525636 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:00.509442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42388 (local address 127.14.241.253:35531)
0928 14:51:00.509629 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:00.509635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:00.510623 (+ 988us) server_negotiation.cc:407] Connection header received
0928 14:51:00.510890 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:00.510895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:00.510961 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:00.511040 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:00.512743 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.513594 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.515491 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.515957 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.516072 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:00.519635 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:00.519644 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:00.519649 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:00.519689 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:00.522684 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:00.523519 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:00.523524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:00.523527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:00.523684 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:00.524069 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:00.524074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:00.524078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:00.524464 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:00.524729 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:00.525085 (+ 356us) server_negotiation.cc:299] Negotiation successful
0928 14:51:00.525315 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:51:00.526000 22767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:00.510628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.14.241.252:42347)
0928 14:51:00.510809 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:00.510815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:00.511048 (+ 233us) server_negotiation.cc:407] Connection header received
0928 14:51:00.511239 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:00.511245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:00.511324 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:00.511423 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:00.513328 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.514193 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.515532 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:00.516153 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:00.516304 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:00.519569 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:00.519574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:00.519579 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:00.519671 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:00.522901 (+ 3230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:00.523628 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:00.523633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:00.523637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:00.523773 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:00.524140 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:00.524145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:00.524148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:00.524498 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:00.524707 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:00.525532 (+ 825us) server_negotiation.cc:299] Negotiation successful
0928 14:51:00.525732 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 14:51:01.300091 22816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.299359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.194:51501)
0928 14:51:01.299807 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.299928 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":212,"threads_started":1}
W20260928 14:51:01.300510 22609 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:33413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
I20260928 14:51:01.301694 22817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.301016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.193:54909)
0928 14:51:01.301434 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.301542 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":210,"threads_started":1}
W20260928 14:51:01.302114 22462 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:33413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
I20260928 14:51:01.305831 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.305150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.195:42815)
0928 14:51:01.305554 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.305662 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":204,"threads_started":1}
W20260928 14:51:01.306175 22750 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:33413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
I20260928 14:51:01.815907 22819 raft_consensus.cc:482] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9917d61e0e9d413b839dfc49f84766a4)
I20260928 14:51:01.816174 22819 raft_consensus.cc:504] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } }
I20260928 14:51:01.817656 22819 leader_election.cc:288] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9917d61e0e9d413b839dfc49f84766a4 (127.14.241.254:33413), 86d4b41acb1a441c87a2f0794cb6ea53 (127.14.241.253:35531)
I20260928 14:51:01.820142 22823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9917d61e0e9d413b839dfc49f84766a4)
I20260928 14:51:01.820516 22823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } }
I20260928 14:51:01.821132 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.817891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56736)
0928 14:51:01.820657 (+ 2766us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.820830 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2658,"mutex_wait_us":40,"thread_start_us":2191,"threads_started":1}
W20260928 14:51:01.821408 22267 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
I20260928 14:51:01.823395 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.822194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56738)
0928 14:51:01.822858 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.823065 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":377,"threads_started":1}
I20260928 14:51:01.823926 22823 leader_election.cc:288] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9917d61e0e9d413b839dfc49f84766a4 (127.14.241.254:33413), d852e652ec6343c69c1e315766983c7b (127.14.241.252:42347)
I20260928 14:51:01.824427 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.823257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56750)
0928 14:51:01.824146 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.824249 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":790}
W20260928 14:51:01.824494 22075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
W20260928 14:51:01.824600 22267 leader_election.cc:334] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9917d61e0e9d413b839dfc49f84766a4 (127.14.241.254:33413): Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
I20260928 14:51:01.826114 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.825749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56754)
0928 14:51:01.825896 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.825981 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260928 14:51:01.826340 22075 leader_election.cc:334] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9917d61e0e9d413b839dfc49f84766a4 (127.14.241.254:33413): Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
I20260928 14:51:01.828801 22822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.817964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42400 (local address 127.14.241.253:35531)
0928 14:51:01.819800 (+ 1836us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:01.819807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:01.821047 (+ 1240us) server_negotiation.cc:407] Connection header received
0928 14:51:01.821247 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:01.821252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:01.821333 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:01.821434 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:01.822817 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.824124 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.826940 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.827879 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.827987 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:01.828037 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:01.828408 (+ 371us) server_negotiation.cc:299] Negotiation successful
0928 14:51:01.828521 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":1712,"thread_start_us":1541,"threads_started":1}
I20260928 14:51:01.828886 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.817891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:35531 (local address 127.0.0.1:42400)
0928 14:51:01.820937 (+ 3046us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.820958 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:01.821103 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:01.821475 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:01.821480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:01.821513 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:01.821841 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:01.821854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.824931 (+ 3077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.824937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:01.826804 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:01.826819 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.828061 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.828067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:01.828233 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:01.828268 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:51:01.828387 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 14:51:01.828509 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2369,"mutex_wait_us":1,"thread_start_us":2485,"threads_started":1}
I20260928 14:51:01.829471 22098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d852e652ec6343c69c1e315766983c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" is_pre_election: true
I20260928 14:51:01.829764 22098 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d852e652ec6343c69c1e315766983c7b in term 1.
I20260928 14:51:01.830258 22268 leader_election.cc:302] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86d4b41acb1a441c87a2f0794cb6ea53, d852e652ec6343c69c1e315766983c7b; no voters: 9917d61e0e9d413b839dfc49f84766a4
I20260928 14:51:01.830643 22825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.822209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42347 (local address 127.0.0.1:45732)
0928 14:51:01.824007 (+ 1798us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.824029 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:01.824162 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:01.825265 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:01.825270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:01.825299 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:01.825663 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:01.825673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.826982 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.826986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:01.828636 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:01.828650 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.829909 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.829915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:01.830091 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:01.830122 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 14:51:01.830212 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 14:51:01.830340 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1654,"thread_start_us":1484,"threads_started":1}
I20260928 14:51:01.830706 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.822441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45732 (local address 127.14.241.252:42347)
0928 14:51:01.824919 (+ 2478us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:01.824925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:01.824945 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:51:01.825006 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:01.825009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:01.825083 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:01.825218 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:01.825824 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.826849 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.828781 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.829737 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.829903 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:01.829972 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:01.830236 (+ 264us) server_negotiation.cc:299] Negotiation successful
0928 14:51:01.830338 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2358,"thread_start_us":2391,"threads_started":1}
I20260928 14:51:01.830688 22819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 14:51:01.830854 22819 raft_consensus.cc:482] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9917d61e0e9d413b839dfc49f84766a4)
I20260928 14:51:01.830947 22819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:51:01.831270 22292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d852e652ec6343c69c1e315766983c7b" is_pre_election: true
I20260928 14:51:01.833200 22819 raft_consensus.cc:504] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } }
I20260928 14:51:01.833617 22292 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 86d4b41acb1a441c87a2f0794cb6ea53 in current term 2: Already voted for candidate d852e652ec6343c69c1e315766983c7b in this term.
I20260928 14:51:01.833912 22819 leader_election.cc:288] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [CANDIDATE]: Term 2 election: Requested vote from peers 9917d61e0e9d413b839dfc49f84766a4 (127.14.241.254:33413), 86d4b41acb1a441c87a2f0794cb6ea53 (127.14.241.253:35531)
I20260928 14:51:01.834411 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.833955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56768)
0928 14:51:01.834195 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.834276 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260928 14:51:01.834372 22098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d852e652ec6343c69c1e315766983c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "86d4b41acb1a441c87a2f0794cb6ea53"
I20260928 14:51:01.834573 22098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:51:01.834652 22076 leader_election.cc:302] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86d4b41acb1a441c87a2f0794cb6ea53; no voters: 9917d61e0e9d413b839dfc49f84766a4, d852e652ec6343c69c1e315766983c7b
I20260928 14:51:01.835386 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.835000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56778)
0928 14:51:01.835142 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.835240 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260928 14:51:01.835552 22267 leader_election.cc:334] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9917d61e0e9d413b839dfc49f84766a4 (127.14.241.254:33413): Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
I20260928 14:51:01.837009 22098 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d852e652ec6343c69c1e315766983c7b in term 2.
I20260928 14:51:01.837170 22823 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:51:01.837386 22268 leader_election.cc:302] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86d4b41acb1a441c87a2f0794cb6ea53, d852e652ec6343c69c1e315766983c7b; no voters: 9917d61e0e9d413b839dfc49f84766a4
I20260928 14:51:01.837709 22819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:51:01.838136 22819 raft_consensus.cc:686] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [term 2 LEADER]: Becoming Leader. State: Replica: d852e652ec6343c69c1e315766983c7b, State: Running, Role: LEADER
I20260928 14:51:01.838500 22819 consensus_queue.cc:237] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } }
I20260928 14:51:01.840304 22829 sys_catalog.cc:455] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d852e652ec6343c69c1e315766983c7b. Latest consensus state: current_term: 2 leader_uuid: "d852e652ec6343c69c1e315766983c7b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:51:01.840701 22829 sys_catalog.cc:458] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:01.841324 22831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:51:01.844837 22831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:51:01.845602 22831 catalog_manager.cc:1259] Loaded cluster ID: a0ac33f33ad647ed9c41bab5190b2b4d
I20260928 14:51:01.845674 22831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:51:01.846781 22831 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:51:01.847524 22831 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Loaded TSK: 0
I20260928 14:51:01.848397 22831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:51:01.862927 22281 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:51:01.870031 22281 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:51:01.874125 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.873711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56794)
0928 14:51:01.873867 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.873979 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260928 14:51:01.874292 22098 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [term 2 FOLLOWER]: Refusing update from remote peer d852e652ec6343c69c1e315766983c7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260928 14:51:01.875068 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.874712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56806)
0928 14:51:01.874866 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.874944 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 14:51:01.875118 22829 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260928 14:51:01.875419 22267 consensus_peers.cc:487] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b -> Peer 9917d61e0e9d413b839dfc49f84766a4 (127.14.241.254:33413): Couldn't send request to peer 9917d61e0e9d413b839dfc49f84766a4. Status: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 14:51:01.877776 22823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d852e652ec6343c69c1e315766983c7b. Latest consensus state: current_term: 2 leader_uuid: "d852e652ec6343c69c1e315766983c7b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:51:01.878027 22823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:51:01.879352 22829 sys_catalog.cc:455] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d852e652ec6343c69c1e315766983c7b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:51:01.879745 22829 sys_catalog.cc:458] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:01.880223 22823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d852e652ec6343c69c1e315766983c7b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9917d61e0e9d413b839dfc49f84766a4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33413 } } peers { permanent_uuid: "86d4b41acb1a441c87a2f0794cb6ea53" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 35531 } attrs { promote: false } } peers { permanent_uuid: "d852e652ec6343c69c1e315766983c7b" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42347 } attrs { promote: false } } }
I20260928 14:51:01.880510 22823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d4b41acb1a441c87a2f0794cb6ea53 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:51:01.894932 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.887549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:39459 (local address 127.0.0.1:59550)
0928 14:51:01.887699 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.887719 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:01.887833 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:01.888567 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:01.888571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:01.888596 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:01.888803 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:01.888814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.890402 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.890408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:01.892592 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:01.892608 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.894104 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.894111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:01.894340 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:01.894393 (+ 53us) client_negotiation.cc:769] Sending connection context
0928 14:51:01.894459 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 14:51:01.894605 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":1}
I20260928 14:51:01.896081 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.887668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.14.241.194:39459)
0928 14:51:01.888246 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:01.888252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:01.888269 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:51:01.888365 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:01.888371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:01.888443 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:01.888533 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:01.888971 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.890243 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.892816 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.893914 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.895497 (+ 1583us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:01.895580 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:01.895633 (+ 53us) server_negotiation.cc:299] Negotiation successful
0928 14:51:01.895777 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":631,"threads_started":1}
I20260928 14:51:01.896478 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.887990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:42739 (local address 127.0.0.1:55550)
0928 14:51:01.888123 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.888138 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:01.888223 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:01.889380 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:01.889385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:01.889412 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:01.889758 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:01.889768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.891347 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.891353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:01.893559 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:01.893576 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.895669 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.895676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:01.895885 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:01.895923 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 14:51:01.895988 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:51:01.896138 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 14:51:01.896772 22542 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab3051d81b94708b036553285c304e9 (DEFAULT_TABLE table=test-table [id=6dc1f8f5f6244c3dbf4d9158fdd93497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:01.897174 22542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ab3051d81b94708b036553285c304e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:01.897367 22839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.888366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55550 (local address 127.14.241.195:42739)
0928 14:51:01.888897 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:01.888903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:01.888922 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:51:01.888995 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:01.889000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:01.889210 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:01.889901 (+ 691us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:01.890066 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.891160 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.893791 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.895419 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.896738 (+ 1319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:01.896820 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:01.896877 (+ 57us) server_negotiation.cc:299] Negotiation successful
0928 14:51:01.897027 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":587,"threads_started":1}
I20260928 14:51:01.898573 22684 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab3051d81b94708b036553285c304e9 (DEFAULT_TABLE table=test-table [id=6dc1f8f5f6244c3dbf4d9158fdd93497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:01.899011 22684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ab3051d81b94708b036553285c304e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:01.904242 22842 tablet_bootstrap.cc:492] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786: Bootstrap starting.
I20260928 14:51:01.905506 22843 tablet_bootstrap.cc:492] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82: Bootstrap starting.
I20260928 14:51:01.906323 22842 tablet_bootstrap.cc:654] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:01.907205 22843 tablet_bootstrap.cc:654] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:01.908351 22842 tablet_bootstrap.cc:492] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786: No bootstrap required, opened a new log
I20260928 14:51:01.908496 22842 ts_tablet_manager.cc:1397] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260928 14:51:01.909186 22843 tablet_bootstrap.cc:492] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82: No bootstrap required, opened a new log
I20260928 14:51:01.909287 22843 ts_tablet_manager.cc:1397] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260928 14:51:01.909246 22842 raft_consensus.cc:348] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } } peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } }
I20260928 14:51:01.909489 22842 raft_consensus.cc:374] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:01.909584 22842 raft_consensus.cc:729] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22174d815bd4fd29ffb608bef8e7786, State: Initialized, Role: FOLLOWER
I20260928 14:51:01.909834 22842 consensus_queue.cc:260] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } } peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } }
I20260928 14:51:01.910038 22843 raft_consensus.cc:348] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } } peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } }
I20260928 14:51:01.910243 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.888724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:44073 (local address 127.0.0.1:34762)
0928 14:51:01.895171 (+ 6447us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.895188 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:01.895286 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:01.904129 (+ 8843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:01.904134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:01.904164 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:01.904396 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:01.904408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.905979 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.905985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:01.907816 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:01.907833 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.909387 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.909394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:01.909614 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:01.909652 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 14:51:01.909760 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 14:51:01.909906 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6348,"thread_start_us":9017,"threads_started":1}
I20260928 14:51:01.910256 22843 raft_consensus.cc:374] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:01.910336 22843 raft_consensus.cc:729] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fc3d05546d4301baae6c6202709f82, State: Initialized, Role: FOLLOWER
I20260928 14:51:01.910570 22843 consensus_queue.cc:260] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } } peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } }
I20260928 14:51:01.911007 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.888818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.14.241.193:44073)
0928 14:51:01.903704 (+ 14886us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:01.903709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:01.903726 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:51:01.903792 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:01.903797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:01.903960 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:01.904543 (+ 583us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:01.904700 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.905822 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.908001 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:01.909201 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:01.909374 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:01.909451 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:01.910570 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0928 14:51:01.910719 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14774,"thread_start_us":14683,"threads_started":1}
I20260928 14:51:01.911723 22842 ts_tablet_manager.cc:1428] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:51:01.911782 22395 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab3051d81b94708b036553285c304e9 (DEFAULT_TABLE table=test-table [id=6dc1f8f5f6244c3dbf4d9158fdd93497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:01.912236 22395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ab3051d81b94708b036553285c304e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:01.915126 22608 heartbeater.cc:503] Master 127.14.241.252:42347 was elected leader, sending a full tablet report...
I20260928 14:51:01.917829 22843 ts_tablet_manager.cc:1428] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82: Time spent starting tablet: real 0.008s user 0.003s sys 0.006s
I20260928 14:51:01.919243 22849 tablet_bootstrap.cc:492] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a: Bootstrap starting.
I20260928 14:51:01.921650 22849 tablet_bootstrap.cc:654] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:01.921710 22749 heartbeater.cc:503] Master 127.14.241.252:42347 was elected leader, sending a full tablet report...
I20260928 14:51:01.923635 22849 tablet_bootstrap.cc:492] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a: No bootstrap required, opened a new log
I20260928 14:51:01.923765 22849 ts_tablet_manager.cc:1397] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260928 14:51:01.924400 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.918847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.195:46291)
0928 14:51:01.923798 (+ 4951us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.924018 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3392,"thread_start_us":3269,"threads_started":1}
I20260928 14:51:01.924518 22849 raft_consensus.cc:348] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } } peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } }
I20260928 14:51:01.924757 22849 raft_consensus.cc:374] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:01.924855 22849 raft_consensus.cc:729] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaef801444514d3eb59008afc430a19a, State: Initialized, Role: FOLLOWER
I20260928 14:51:01.925240 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.912189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.194:59891)
0928 14:51:01.924805 (+ 12616us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.924976 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12471,"thread_start_us":12299,"threads_started":1}
I20260928 14:51:01.925124 22849 consensus_queue.cc:260] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } } peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } }
I20260928 14:51:01.926559 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.926120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.194:60457)
0928 14:51:01.926325 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.926419 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260928 14:51:01.927099 22849 ts_tablet_manager.cc:1428] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:51:01.927610 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.926117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.193:36613)
0928 14:51:01.927157 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.927347 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":1282,"threads_started":1}
I20260928 14:51:01.928541 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:01.928137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.193:32803)
0928 14:51:01.928273 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:01.928399 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 14:51:01.928611 22461 heartbeater.cc:503] Master 127.14.241.252:42347 was elected leader, sending a full tablet report...
I20260928 14:51:02.113209 22852 raft_consensus.cc:482] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:51:02.113447 22852 raft_consensus.cc:504] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } } peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } }
I20260928 14:51:02.114967 22852 leader_election.cc:288] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c22174d815bd4fd29ffb608bef8e7786 (127.14.241.194:39459), b0fc3d05546d4301baae6c6202709f82 (127.14.241.195:42739)
I20260928 14:51:02.121042 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.115423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:37459 (local address 127.14.241.194:39459)
0928 14:51:02.115625 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:02.115630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:02.115649 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:51:02.115909 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:02.115915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:02.115993 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:02.116100 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:02.116709 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.117508 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.119297 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.120099 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.120216 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:02.120272 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:02.120664 (+ 392us) server_negotiation.cc:299] Negotiation successful
0928 14:51:02.120793 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:51:02.121169 22839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.115421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:36825 (local address 127.14.241.195:42739)
0928 14:51:02.115617 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:02.115624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:02.116116 (+ 492us) server_negotiation.cc:407] Connection header received
0928 14:51:02.116311 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:02.116318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:02.116394 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:02.116489 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:02.116999 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.117707 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.119365 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.120256 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.120448 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:02.120517 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:02.120814 (+ 297us) server_negotiation.cc:299] Negotiation successful
0928 14:51:02.120945 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 14:51:02.121204 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.115225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:39459 (local address 127.14.241.193:37459)
0928 14:51:02.115547 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:02.115567 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:02.115748 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:02.116145 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:02.116150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:02.116177 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:02.116538 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:02.116551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.117631 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.117636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:02.119126 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:02.119139 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.120260 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.120266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:02.120499 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:02.120536 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 14:51:02.120614 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:51:02.120759 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":3}
I20260928 14:51:02.121552 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.115341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:42739 (local address 127.14.241.193:36825)
0928 14:51:02.116015 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:02.116032 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:02.116151 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:02.116506 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:02.116509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:02.116531 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:02.116855 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:02.116866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.117834 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.117839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:02.119216 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:02.119230 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.120471 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.120476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:02.120615 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:02.120646 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 14:51:02.120773 (+ 127us) client_negotiation.cc:240] Negotiation successful
0928 14:51:02.120903 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":3,"thread_start_us":404,"threads_started":1}
I20260928 14:51:02.121848 22563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab3051d81b94708b036553285c304e9" candidate_uuid: "aaef801444514d3eb59008afc430a19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22174d815bd4fd29ffb608bef8e7786" is_pre_election: true
I20260928 14:51:02.122097 22563 raft_consensus.cc:2454] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaef801444514d3eb59008afc430a19a in term 0.
I20260928 14:51:02.122450 22704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab3051d81b94708b036553285c304e9" candidate_uuid: "aaef801444514d3eb59008afc430a19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fc3d05546d4301baae6c6202709f82" is_pre_election: true
I20260928 14:51:02.122874 22704 raft_consensus.cc:2454] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaef801444514d3eb59008afc430a19a in term 0.
I20260928 14:51:02.122841 22350 leader_election.cc:302] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaef801444514d3eb59008afc430a19a, c22174d815bd4fd29ffb608bef8e7786; no voters:
I20260928 14:51:02.123291 22852 raft_consensus.cc:2781] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:51:02.123425 22852 raft_consensus.cc:482] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:51:02.123521 22852 raft_consensus.cc:3037] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:02.126459 22852 raft_consensus.cc:504] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } } peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } }
I20260928 14:51:02.127038 22852 leader_election.cc:288] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [CANDIDATE]: Term 1 election: Requested vote from peers c22174d815bd4fd29ffb608bef8e7786 (127.14.241.194:39459), b0fc3d05546d4301baae6c6202709f82 (127.14.241.195:42739)
I20260928 14:51:02.127476 22704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab3051d81b94708b036553285c304e9" candidate_uuid: "aaef801444514d3eb59008afc430a19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fc3d05546d4301baae6c6202709f82"
I20260928 14:51:02.127483 22563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab3051d81b94708b036553285c304e9" candidate_uuid: "aaef801444514d3eb59008afc430a19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22174d815bd4fd29ffb608bef8e7786"
I20260928 14:51:02.127640 22704 raft_consensus.cc:3037] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:02.127653 22563 raft_consensus.cc:3037] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:02.129611 22563 raft_consensus.cc:2454] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaef801444514d3eb59008afc430a19a in term 1.
I20260928 14:51:02.129688 22704 raft_consensus.cc:2454] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaef801444514d3eb59008afc430a19a in term 1.
I20260928 14:51:02.130082 22349 leader_election.cc:302] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaef801444514d3eb59008afc430a19a, b0fc3d05546d4301baae6c6202709f82; no voters:
I20260928 14:51:02.130357 22852 raft_consensus.cc:2781] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:02.130765 22852 raft_consensus.cc:686] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 1 LEADER]: Becoming Leader. State: Replica: aaef801444514d3eb59008afc430a19a, State: Running, Role: LEADER
I20260928 14:51:02.131183 22852 consensus_queue.cc:237] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } } peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } }
I20260928 14:51:02.131733 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.131300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.193:47791)
0928 14:51:02.131481 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:02.131589 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260928 14:51:02.134594 22281 catalog_manager.cc:5495] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a reported cstate change: term changed from 0 to 1, leader changed from <none> to aaef801444514d3eb59008afc430a19a (127.14.241.193). New cstate: current_term: 1 leader_uuid: "aaef801444514d3eb59008afc430a19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaef801444514d3eb59008afc430a19a" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44073 } health_report { overall_health: HEALTHY } } }
I20260928 14:51:02.194653 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.189607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34776 (local address 127.14.241.193:44073)
0928 14:51:02.189827 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:02.189833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:02.189853 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:51:02.189962 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:02.189966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:02.190048 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:02.190173 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:51:02.190518 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.191205 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.192587 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.193010 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.193129 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:02.193523 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:51:02.194075 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:51:02.194206 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:02.194308 (+ 102us) server_negotiation.cc:299] Negotiation successful
0928 14:51:02.194389 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 14:51:02.198817 22563 raft_consensus.cc:1261] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786 [term 1 FOLLOWER]: Refusing update from remote peer aaef801444514d3eb59008afc430a19a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:02.199121 22704 raft_consensus.cc:1261] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82 [term 1 FOLLOWER]: Refusing update from remote peer aaef801444514d3eb59008afc430a19a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:02.199548 22852 consensus_queue.cc:1035] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c22174d815bd4fd29ffb608bef8e7786" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:02.199983 22855 consensus_queue.cc:1035] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fc3d05546d4301baae6c6202709f82" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:02.203292 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.202155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.194:55393)
0928 14:51:02.202922 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:02.203076 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":620}
I20260928 14:51:02.204109 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.203726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.195:58825)
0928 14:51:02.203860 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:02.203957 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 14:51:02.219532 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.219057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.194:55289)
0928 14:51:02.219267 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:02.219361 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260928 14:51:02.232224 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.231606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.193:57403)
0928 14:51:02.231905 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:02.232010 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260928 14:51:02.232700 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.231621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.195:42265)
0928 14:51:02.232444 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:02.232540 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":704}
I20260928 14:51:02.236083 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.235657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.14.241.193:58279)
0928 14:51:02.235852 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:02.235939 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260928 14:51:02.244020 22281 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:02.244336 22281 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:02.250200 22281 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d852e652ec6343c69c1e315766983c7b: Sending DeleteTablet for 3 replicas of tablet 6ab3051d81b94708b036553285c304e9
I20260928 14:51:02.251164 22542 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ab3051d81b94708b036553285c304e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:02 UTC) from {username='oryx', principal='oryx/127.14.241.252@KRBTEST.COM'} at 127.0.0.1:59550
I20260928 14:51:02.251284 22684 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ab3051d81b94708b036553285c304e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:02 UTC) from {username='oryx', principal='oryx/127.14.241.252@KRBTEST.COM'} at 127.0.0.1:55550
I20260928 14:51:02.251509 22395 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ab3051d81b94708b036553285c304e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:02 UTC) from {username='oryx', principal='oryx/127.14.241.252@KRBTEST.COM'} at 127.0.0.1:34762
I20260928 14:51:02.253741 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 22253
I20260928 14:51:02.261886 22870 tablet_replica.cc:330] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a: stopping tablet replica
I20260928 14:51:02.262262 22870 raft_consensus.cc:2229] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:51:02.262997 22870 raft_consensus.cc:2258] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:02.264338 22870 ts_tablet_manager.cc:1905] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:02.272037 22870 ts_tablet_manager.cc:1918] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:02.272747 22868 tablet_replica.cc:330] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786: stopping tablet replica
I20260928 14:51:02.272984 22868 raft_consensus.cc:2229] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:02.273327 22868 raft_consensus.cc:2258] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:02.273605 22870 log.cc:1196] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/6ab3051d81b94708b036553285c304e9
I20260928 14:51:02.274375 22869 tablet_replica.cc:330] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82: stopping tablet replica
I20260928 14:51:02.274551 22869 raft_consensus.cc:2229] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:02.274825 22869 raft_consensus.cc:2258] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:02.275768 22868 ts_tablet_manager.cc:1905] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:02.275768 22869 ts_tablet_manager.cc:1905] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:02.282300 22868 ts_tablet_manager.cc:1918] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:02.282300 22869 ts_tablet_manager.cc:1918] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:02.282436 22869 log.cc:1196] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/6ab3051d81b94708b036553285c304e9
I20260928 14:51:02.282809 22870 ts_tablet_manager.cc:1939] T 6ab3051d81b94708b036553285c304e9 P aaef801444514d3eb59008afc430a19a: Deleting consensus metadata
I20260928 14:51:02.282869 22869 ts_tablet_manager.cc:1939] T 6ab3051d81b94708b036553285c304e9 P b0fc3d05546d4301baae6c6202709f82: Deleting consensus metadata
I20260928 14:51:02.284010 22868 log.cc:1196] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/6ab3051d81b94708b036553285c304e9
I20260928 14:51:02.284060 22266 catalog_manager.cc:4841] TS aaef801444514d3eb59008afc430a19a (127.14.241.193:44073): tablet 6ab3051d81b94708b036553285c304e9 (table test-table [id=6dc1f8f5f6244c3dbf4d9158fdd93497]) successfully deleted
I20260928 14:51:02.284416 22266 catalog_manager.cc:4841] TS b0fc3d05546d4301baae6c6202709f82 (127.14.241.195:42739): tablet 6ab3051d81b94708b036553285c304e9 (table test-table [id=6dc1f8f5f6244c3dbf4d9158fdd93497]) successfully deleted
I20260928 14:51:02.284731 22868 ts_tablet_manager.cc:1939] T 6ab3051d81b94708b036553285c304e9 P c22174d815bd4fd29ffb608bef8e7786: Deleting consensus metadata
I20260928 14:51:02.285624 22267 catalog_manager.cc:4841] TS c22174d815bd4fd29ffb608bef8e7786 (127.14.241.194:39459): tablet 6ab3051d81b94708b036553285c304e9 (table test-table [id=6dc1f8f5f6244c3dbf4d9158fdd93497]) successfully deleted
I20260928 14:51:02.286592 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.255560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.14.241.193:44073)
0928 14:51:02.255761 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:02.255768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:02.268912 (+ 13144us) server_negotiation.cc:407] Connection header received
0928 14:51:02.269253 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:02.269260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:02.269396 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:02.269477 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:02.271616 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.272772 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.274424 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.275069 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.275195 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:02.279074 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:02.279096 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:02.279117 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:02.279162 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:02.283185 (+ 4023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:02.284104 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:02.284113 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:02.284117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:02.284321 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:02.284770 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:02.284775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:02.284778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:02.285231 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:02.285536 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:02.286085 (+ 549us) server_negotiation.cc:299] Negotiation successful
0928 14:51:02.286274 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:51:02.287319 22456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:02.390811 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 22253
I20260928 14:51:02.405644 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 22473
I20260928 14:51:02.418572 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.406337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.14.241.194:39459)
0928 14:51:02.406490 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:02.406494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:02.406533 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:51:02.406694 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:02.406698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:02.406763 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:02.406885 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:02.408089 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.408917 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.410199 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.410667 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.410783 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:02.413435 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:02.413452 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:02.413469 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:02.413508 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:02.416158 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:02.416826 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:02.416833 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:02.416836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:02.416985 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:02.417259 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:02.417262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:02.417264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:02.417621 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:02.417834 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:02.418036 (+ 202us) server_negotiation.cc:299] Negotiation successful
0928 14:51:02.418231 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:51:02.419221 22603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:02.419986 22873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.419146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56814)
0928 14:51:02.419643 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:02.419753 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":228,"threads_started":1}
I20260928 14:51:02.421484 22873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.421108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33413 (local address 127.0.0.1:56816)
0928 14:51:02.421251 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:02.421339 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260928 14:51:02.515172 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 22473
I20260928 14:51:02.529345 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 22621
I20260928 14:51:02.542203 22839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:02.530065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.14.241.195:42739)
0928 14:51:02.530238 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:02.530243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:02.530264 (+ 21us) server_negotiation.cc:407] Connection header received
0928 14:51:02.530426 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:02.530432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:02.530503 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:02.530584 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:02.531871 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.532641 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.533859 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:02.534228 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:02.534331 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:02.536737 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:02.536752 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:02.536768 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:02.536800 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:02.539377 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:02.540016 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:02.540024 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:02.540026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:02.540153 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:02.540690 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:02.540696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:02.540699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:02.541076 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:02.541380 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:02.541663 (+ 283us) server_negotiation.cc:299] Negotiation successful
0928 14:51:02.541888 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 14:51:02.542830 22744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:02.661219 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 22621
I20260928 14:51:02.678961 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 21950
I20260928 14:51:02.680122 22119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:02.796648 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 21950
I20260928 14:51:02.813215 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 22053
I20260928 14:51:02.814260 22310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:02.933912 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 22053
2026-09-28T14:51:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9503 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:51:03 dist-test-slave-vk40 krb5kdc[22882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:51:03 dist-test-slave-vk40 krb5kdc[22882](info): set up 2 sockets
Sep 28 14:51:03 dist-test-slave-vk40 krb5kdc[22882](info): commencing operation
krb5kdc: starting...
W20260928 14:51:03.936151 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:51:04 dist-test-slave-vk40 krb5kdc[22882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:51:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:51:04Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.254.keytab.
Entry for principal oryx/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.254.keytab.
I20260928 14:51:04.152266 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:45975
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.254.keytab
--principal=oryx/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:45975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:04.486569 22896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:04.486936 22896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:04.487085 22896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:04.494431 22896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:51:04.494558 22896 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 14:51:04.494623 22896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:04.494710 22896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:04.494778 22896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:51:04.494869 22896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:51:04.504977 22896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:45975
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.254.keytab
--principal=oryx/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:45975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:04.506304 22896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:04.507673 22896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:04.519564 22901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:04.519573 22902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:04.519546 22904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:04.520140 22896 server_base.cc:1034] running on GCE node
I20260928 14:51:04.521134 22896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:04.522688 22896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:04.523917 22896 hybrid_clock.cc:648] HybridClock initialized: now 1790607064523847 us; error 46 us; skew 500 ppm
I20260928 14:51:04.524248 22896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:04 dist-test-slave-vk40 krb5kdc[22882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607064, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:04.527380 22896 init.cc:377] Logged in from keytab as oryx/127.14.241.254@KRBTEST.COM (short username oryx)
I20260928 14:51:04.528956 22896 webserver.cc:460] Webserver started at http://127.14.241.254:39265/ using document root <none> and password file <none>
I20260928 14:51:04.529695 22896 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:04.529800 22896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:04.530121 22896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:04.533207 22896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "7326e50052ad478fabe0270bb875996b"
format_stamp: "Formatted at 2026-09-28 14:51:04 on dist-test-slave-vk40"
server_key: "2cd7882f9c0aa0c887f0fe35e105ae8a"
server_key_iv: "3ac2fc4addbb93509ceba6393bd42581"
server_key_version: "encryptionkey@0"
I20260928 14:51:04.534019 22896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "7326e50052ad478fabe0270bb875996b"
format_stamp: "Formatted at 2026-09-28 14:51:04 on dist-test-slave-vk40"
server_key: "2cd7882f9c0aa0c887f0fe35e105ae8a"
server_key_iv: "3ac2fc4addbb93509ceba6393bd42581"
server_key_version: "encryptionkey@0"
I20260928 14:51:04.539391 22896 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260928 14:51:04.543365 22911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:04.544921 22896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:51:04.545102 22896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "7326e50052ad478fabe0270bb875996b"
format_stamp: "Formatted at 2026-09-28 14:51:04 on dist-test-slave-vk40"
server_key: "2cd7882f9c0aa0c887f0fe35e105ae8a"
server_key_iv: "3ac2fc4addbb93509ceba6393bd42581"
server_key_version: "encryptionkey@0"
I20260928 14:51:04.545312 22896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:04.568929 22896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:04.573081 22896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:04.573340 22896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:04.593367 22896 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:45975
I20260928 14:51:04.593391 22972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:45975 every 8 connection(s)
I20260928 14:51:04.594719 22896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:51:04.598752 22973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:04.602869 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 22896
I20260928 14:51:04.603044 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:51:04.603713 15303 external_mini_cluster.cc:1428] Setting key 06fda205b6208ae2addad41fcb2f84a0
I20260928 14:51:04.609766 22973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b: Bootstrap starting.
I20260928 14:51:04.613634 22973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:04.614835 22973 log.cc:824] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b: Log is configured to *not* fsync() on all Append() calls
Sep 28 14:51:04 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:51:04.618139 22973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b: No bootstrap required, opened a new log
I20260928 14:51:04.626893 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:04.605761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.14.241.254:45975)
0928 14:51:04.606568 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:04.606580 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:04.606621 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:51:04.608445 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:04.608469 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:04.608875 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:04.609189 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:04.610897 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:04.612222 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:04.613985 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:04.614705 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:04.614865 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:04.619267 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:04.619275 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:04.619291 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:04.619334 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:04.622351 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:04.623272 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:04.623280 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:04.623287 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:04.623579 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:04.623998 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:04.624002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:04.624004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:04.624649 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:04.624881 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:04.625333 (+ 452us) server_negotiation.cc:299] Negotiation successful
0928 14:51:04.625760 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":343,"threads_started":1}
I20260928 14:51:04.627065 22973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7326e50052ad478fabe0270bb875996b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45975 } }
I20260928 14:51:04.627476 22973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:04.627583 22973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7326e50052ad478fabe0270bb875996b, State: Initialized, Role: FOLLOWER
I20260928 14:51:04.628198 22973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7326e50052ad478fabe0270bb875996b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45975 } }
I20260928 14:51:04.628504 22973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:51:04.628633 22973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:51:04.628808 22973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:04.631160 22973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7326e50052ad478fabe0270bb875996b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45975 } }
I20260928 14:51:04.631637 22973 leader_election.cc:302] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7326e50052ad478fabe0270bb875996b; no voters:
I20260928 14:51:04.632166 22973 leader_election.cc:288] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:51:04.632316 22978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:04.633042 22978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [term 1 LEADER]: Becoming Leader. State: Replica: 7326e50052ad478fabe0270bb875996b, State: Running, Role: LEADER
I20260928 14:51:04.633606 22978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7326e50052ad478fabe0270bb875996b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45975 } }
I20260928 14:51:04.634192 22973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:51:04.636502 22980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7326e50052ad478fabe0270bb875996b. Latest consensus state: current_term: 1 leader_uuid: "7326e50052ad478fabe0270bb875996b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7326e50052ad478fabe0270bb875996b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45975 } } }
I20260928 14:51:04.636950 22980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:04.636830 22979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7326e50052ad478fabe0270bb875996b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7326e50052ad478fabe0270bb875996b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45975 } } }
I20260928 14:51:04.637338 22979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:04.638062 22985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:51:04.643903 22985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:51:04.653126 22985 catalog_manager.cc:1347] Generated new cluster ID: fa9352f82fc64c5384a61a26ac95501b
I20260928 14:51:04.653252 22985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:51:04.694413 22985 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:51:04.695992 22985 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:51:04.707361 22985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b: Generated new TSK 0
I20260928 14:51:04.708370 22985 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab.
Entry for principal oryx/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab.
I20260928 14:51:04.772920 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:45975
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab
--principal=oryx/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:05.096594 22999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:05.096872 22999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:05.097008 22999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:05.104151 22999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:05.104257 22999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:05.104395 22999 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:51:05.104499 22999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:51:05.116246 22999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab
--principal=oryx/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.241.254:45975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:05.117983 22999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:05.119721 22999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:05.137035 23007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:05.137033 23005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:05.136950 23004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:05.139094 22999 server_base.cc:1034] running on GCE node
I20260928 14:51:05.139907 22999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:05.141191 22999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:05.142434 22999 hybrid_clock.cc:648] HybridClock initialized: now 1790607065142355 us; error 51 us; skew 500 ppm
I20260928 14:51:05.142800 22999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:05 dist-test-slave-vk40 krb5kdc[22882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607065, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:05.146795 22999 init.cc:377] Logged in from keytab as oryx/127.14.241.193@KRBTEST.COM (short username oryx)
I20260928 14:51:05.148463 22999 webserver.cc:460] Webserver started at http://127.14.241.193:37219/ using document root <none> and password file <none>
I20260928 14:51:05.149196 22999 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:05.149286 22999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:05.149549 22999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:05.152536 22999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "a93577de632b4d6d919d2c5d9461e6a2"
format_stamp: "Formatted at 2026-09-28 14:51:05 on dist-test-slave-vk40"
server_key: "bafc41d255e787e68135a35c53845d15"
server_key_iv: "62dbbf476a68ae35aabb5ff91d8f5742"
server_key_version: "encryptionkey@0"
I20260928 14:51:05.153291 22999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "a93577de632b4d6d919d2c5d9461e6a2"
format_stamp: "Formatted at 2026-09-28 14:51:05 on dist-test-slave-vk40"
server_key: "bafc41d255e787e68135a35c53845d15"
server_key_iv: "62dbbf476a68ae35aabb5ff91d8f5742"
server_key_version: "encryptionkey@0"
I20260928 14:51:05.158349 22999 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:51:05.161875 23014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:05.163298 22999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:51:05.163447 22999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "a93577de632b4d6d919d2c5d9461e6a2"
format_stamp: "Formatted at 2026-09-28 14:51:05 on dist-test-slave-vk40"
server_key: "bafc41d255e787e68135a35c53845d15"
server_key_iv: "62dbbf476a68ae35aabb5ff91d8f5742"
server_key_version: "encryptionkey@0"
I20260928 14:51:05.163642 22999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:05.183158 22999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:05.187088 22999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:05.187310 22999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:05.190014 22999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:05.190115 22999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:05.190248 22999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:05.190323 22999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:51:05 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607065, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:51:05.212723 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:05.190493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:60821 (local address 127.14.241.254:45975)
0928 14:51:05.191177 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:05.191182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:05.192122 (+ 940us) server_negotiation.cc:407] Connection header received
0928 14:51:05.194164 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:05.194170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:05.194253 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:05.194345 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:05.196595 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.197779 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.200255 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.200866 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.201006 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:05.205801 (+ 4795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:05.205806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:05.205809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:05.205854 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:05.209350 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:05.210165 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:05.210171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:05.210176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:05.210353 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:05.210763 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:05.210768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:05.210772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:05.211152 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:05.211319 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:05.212006 (+ 687us) server_negotiation.cc:299] Negotiation successful
0928 14:51:05.212319 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":485,"threads_started":1}
I20260928 14:51:05.214020 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:05.191160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:45975 (local address 127.14.241.193:60821)
0928 14:51:05.191959 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:05.192007 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:05.193891 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:05.194628 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:05.194638 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:05.195478 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:05.196370 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:05.196404 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.197965 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.197970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:05.200086 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:05.200104 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.201140 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.201154 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:05.201515 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:05.202384 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:05.202416 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:05.205556 (+ 3140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:05.209534 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:05.209546 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:05.209565 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:05.210025 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:05.210451 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:05.210454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:05.210456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:05.210664 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:05.211261 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:05.211267 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:05.211649 (+ 382us) client_negotiation.cc:769] Sending connection context
0928 14:51:05.211886 (+ 237us) client_negotiation.cc:240] Negotiation successful
0928 14:51:05.212368 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":330,"threads_started":1}
I20260928 14:51:05.237125 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:05.223089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:35087 (local address 127.14.241.254:45975)
0928 14:51:05.223306 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:05.223313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:05.223847 (+ 534us) server_negotiation.cc:407] Connection header received
0928 14:51:05.224054 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:05.224058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:05.224127 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:05.224209 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:05.225813 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.226533 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.228617 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.229108 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.229237 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:05.231534 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:05.231539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:05.231542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:05.231577 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:05.234444 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:05.235084 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:05.235088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:05.235091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:05.235224 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:05.235629 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:05.235634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:05.235638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:05.235967 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:05.236093 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:05.236557 (+ 464us) server_negotiation.cc:299] Negotiation successful
0928 14:51:05.236769 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:51:05.237222 23100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:05.223064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:45975 (local address 127.14.241.193:35087)
0928 14:51:05.223726 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:05.223748 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:05.223926 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:05.224258 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:05.224263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:05.225009 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:05.225577 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:05.225595 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.226677 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.226683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:05.228451 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:05.228466 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.229502 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.229512 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:05.229734 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:05.230372 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:05.230392 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:05.231299 (+ 907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:05.234583 (+ 3284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:05.234589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:05.234593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:05.234970 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:05.235345 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:05.235351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:05.235355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:05.235526 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:05.236075 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:05.236080 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:05.236259 (+ 179us) client_negotiation.cc:769] Sending connection context
0928 14:51:05.236446 (+ 187us) client_negotiation.cc:240] Negotiation successful
0928 14:51:05.236798 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":332,"threads_started":1}
I20260928 14:51:05.246510 22999 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:44941
I20260928 14:51:05.246532 23134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:44941 every 8 connection(s)
I20260928 14:51:05.247758 22999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:51:05.249957 23135 heartbeater.cc:347] Connected to a master server at 127.14.241.254:45975
I20260928 14:51:05.250180 23135 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:05.250818 23135 heartbeater.cc:511] Master 127.14.241.254:45975 requested a full tablet report, sending...
I20260928 14:51:05.252786 22928 ts_manager.cc:194] Registered new tserver with Master: a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941)
I20260928 14:51:05.254240 22928 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:60821
I20260928 14:51:05.256639 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 22999
I20260928 14:51:05.256845 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:51:05.257496 15303 external_mini_cluster.cc:1428] Setting key 90d66bf87fcdadccab1f897679ae773f
WARNING: no policy specified for oryx/127.14.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab.
Entry for principal oryx/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab.
I20260928 14:51:05.303345 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:45975
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab
--principal=oryx/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:05.622116 23139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:05.622390 23139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:05.622499 23139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:05.629623 23139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:05.629736 23139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:05.629827 23139 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:51:05.629902 23139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:51:05.639977 23139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab
--principal=oryx/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.241.254:45975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:05.641366 23139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:05.642650 23139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:05.655241 23144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:05.655295 23145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:05.655208 23147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:05.655949 23139 server_base.cc:1034] running on GCE node
I20260928 14:51:05.656602 23139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:05.657490 23139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:05.658707 23139 hybrid_clock.cc:648] HybridClock initialized: now 1790607065658641 us; error 47 us; skew 500 ppm
I20260928 14:51:05.658995 23139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:05 dist-test-slave-vk40 krb5kdc[22882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607065, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:05.666299 23139 init.cc:377] Logged in from keytab as oryx/127.14.241.194@KRBTEST.COM (short username oryx)
I20260928 14:51:05.667930 23139 webserver.cc:460] Webserver started at http://127.14.241.194:35199/ using document root <none> and password file <none>
I20260928 14:51:05.668686 23139 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:05.668776 23139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:05.669031 23139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:05.671871 23139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "d83c56e4461d4027addc640ce7929367"
format_stamp: "Formatted at 2026-09-28 14:51:05 on dist-test-slave-vk40"
server_key: "bffe788c009aaf6e63c312e9c13a484d"
server_key_iv: "b9b8e7c2b40b66d000c4b4d862ddf443"
server_key_version: "encryptionkey@0"
I20260928 14:51:05.672652 23139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "d83c56e4461d4027addc640ce7929367"
format_stamp: "Formatted at 2026-09-28 14:51:05 on dist-test-slave-vk40"
server_key: "bffe788c009aaf6e63c312e9c13a484d"
server_key_iv: "b9b8e7c2b40b66d000c4b4d862ddf443"
server_key_version: "encryptionkey@0"
I20260928 14:51:05.677652 23139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:51:05.681102 23154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:05.682586 23139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:51:05.682757 23139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "d83c56e4461d4027addc640ce7929367"
format_stamp: "Formatted at 2026-09-28 14:51:05 on dist-test-slave-vk40"
server_key: "bffe788c009aaf6e63c312e9c13a484d"
server_key_iv: "b9b8e7c2b40b66d000c4b4d862ddf443"
server_key_version: "encryptionkey@0"
I20260928 14:51:05.682966 23139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:05.702534 23139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:05.706635 23139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:05.706911 23139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:05.709836 23139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:05.709952 23139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:05.710093 23139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:05.710156 23139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:51:05 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607065, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:51:05.732896 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:05.710206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:47879 (local address 127.14.241.254:45975)
0928 14:51:05.710373 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:05.710379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:05.711898 (+ 1519us) server_negotiation.cc:407] Connection header received
0928 14:51:05.714019 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:05.714025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:05.714114 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:05.714224 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:05.716833 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.717858 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.720235 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.720848 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.720964 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:05.725602 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:05.725610 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:05.725616 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:05.725665 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:05.729151 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:05.730150 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:05.730156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:05.730159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:05.730326 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:05.730793 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:05.730799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:05.730803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:05.731201 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:05.731368 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:05.732328 (+ 960us) server_negotiation.cc:299] Negotiation successful
0928 14:51:05.732559 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:51:05.733958 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:05.710876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:45975 (local address 127.14.241.194:47879)
0928 14:51:05.711723 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:05.711770 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:05.713731 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:05.714560 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:05.714574 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:05.715555 (+ 981us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:05.716601 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:05.716631 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.718017 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.718026 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:05.720070 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:05.720082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.721096 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.721104 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:05.721416 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:05.722171 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:05.722207 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:05.725318 (+ 3111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:05.729356 (+ 4038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:05.729370 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:05.729388 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:05.729982 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:05.730466 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:05.730469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:05.730472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:05.730667 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:05.731357 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:05.731367 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:05.731857 (+ 490us) client_negotiation.cc:769] Sending connection context
0928 14:51:05.732187 (+ 330us) client_negotiation.cc:240] Negotiation successful
0928 14:51:05.732767 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":335,"threads_started":1}
I20260928 14:51:05.757018 23139 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:43229
I20260928 14:51:05.757155 23273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:43229 every 8 connection(s)
I20260928 14:51:05.758315 23139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:51:05.760387 23274 heartbeater.cc:347] Connected to a master server at 127.14.241.254:45975
I20260928 14:51:05.760597 23274 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:05.761437 23274 heartbeater.cc:511] Master 127.14.241.254:45975 requested a full tablet report, sending...
I20260928 14:51:05.761551 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:05.747325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:36977 (local address 127.14.241.254:45975)
0928 14:51:05.747560 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:05.747566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:05.748186 (+ 620us) server_negotiation.cc:407] Connection header received
0928 14:51:05.748467 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:05.748472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:05.748546 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:05.748625 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:05.750117 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.750884 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.753175 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.753715 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.753829 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:05.755889 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:05.755893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:05.755897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:05.755929 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:05.758760 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:05.759620 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:05.759625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:05.759629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:05.759768 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:05.760134 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:05.760138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:05.760140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:05.760587 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:05.760719 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:05.761158 (+ 439us) server_negotiation.cc:299] Negotiation successful
0928 14:51:05.761321 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 14:51:05.761849 23249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:05.747420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:45975 (local address 127.14.241.194:36977)
0928 14:51:05.748073 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:05.748102 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:05.748311 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:05.748682 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:05.748687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:05.749444 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:05.749951 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:05.749970 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.751024 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.751030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:05.752945 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:05.752966 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:05.754065 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:05.754071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:05.754252 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:05.754975 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:05.754995 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:05.755708 (+ 713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:05.758895 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:05.758900 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:05.758908 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:05.759472 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:05.759883 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:05.759886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:05.759889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:05.760027 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:05.760717 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:05.760722 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:05.760924 (+ 202us) client_negotiation.cc:769] Sending connection context
0928 14:51:05.761082 (+ 158us) client_negotiation.cc:240] Negotiation successful
0928 14:51:05.761430 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":358,"threads_started":1}
I20260928 14:51:05.762933 22927 ts_manager.cc:194] Registered new tserver with Master: d83c56e4461d4027addc640ce7929367 (127.14.241.194:43229)
I20260928 14:51:05.763079 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 23139
I20260928 14:51:05.763250 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:51:05.763803 15303 external_mini_cluster.cc:1428] Setting key 95d452a62ab0854449e938c3eb106267
I20260928 14:51:05.763857 22927 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:47879
WARNING: no policy specified for oryx/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab.
Entry for principal oryx/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab.
I20260928 14:51:05.807999 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:45975
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab
--principal=oryx/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:06.127856 23278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:06.128149 23278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:06.128311 23278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:06.136134 23278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:06.136293 23278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:06.136447 23278 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:51:06.136569 23278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:51:06.147286 23278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab
--principal=oryx/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.241.254:45975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:06.148591 23278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:06.149982 23278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:06.163353 23286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:06.163312 23284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:06.163409 23283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:06.163657 23278 server_base.cc:1034] running on GCE node
I20260928 14:51:06.164510 23278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:06.165457 23278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:06.166697 23278 hybrid_clock.cc:648] HybridClock initialized: now 1790607066166623 us; error 39 us; skew 500 ppm
I20260928 14:51:06.167001 23278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:06 dist-test-slave-vk40 krb5kdc[22882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607066, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:06.170056 23278 init.cc:377] Logged in from keytab as oryx/127.14.241.195@KRBTEST.COM (short username oryx)
I20260928 14:51:06.171510 23278 webserver.cc:460] Webserver started at http://127.14.241.195:38889/ using document root <none> and password file <none>
I20260928 14:51:06.172191 23278 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:06.172323 23278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:06.172628 23278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:06.175539 23278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "e2e457bb98dd478caa3cccb3bfd88a4b"
format_stamp: "Formatted at 2026-09-28 14:51:06 on dist-test-slave-vk40"
server_key: "8dbdb464ca1152510cc1c5c9e2c55585"
server_key_iv: "7b3d06f337a88a00b02aed274c08f7ba"
server_key_version: "encryptionkey@0"
I20260928 14:51:06.176391 23278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "e2e457bb98dd478caa3cccb3bfd88a4b"
format_stamp: "Formatted at 2026-09-28 14:51:06 on dist-test-slave-vk40"
server_key: "8dbdb464ca1152510cc1c5c9e2c55585"
server_key_iv: "7b3d06f337a88a00b02aed274c08f7ba"
server_key_version: "encryptionkey@0"
I20260928 14:51:06.181793 23278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260928 14:51:06.185638 23293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:06.187211 23278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:51:06.187357 23278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "e2e457bb98dd478caa3cccb3bfd88a4b"
format_stamp: "Formatted at 2026-09-28 14:51:06 on dist-test-slave-vk40"
server_key: "8dbdb464ca1152510cc1c5c9e2c55585"
server_key_iv: "7b3d06f337a88a00b02aed274c08f7ba"
server_key_version: "encryptionkey@0"
I20260928 14:51:06.187553 23278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:06.203611 23278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:06.207792 23278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:06.208021 23278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:06.210602 23278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:06.210706 23278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:06.210834 23278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:06.210899 23278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:51:06 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607066, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:51:06.234242 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.211364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:57805 (local address 127.14.241.254:45975)
0928 14:51:06.211561 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:06.211568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:06.213094 (+ 1526us) server_negotiation.cc:407] Connection header received
0928 14:51:06.215415 (+ 2321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:06.215423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:06.215524 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:06.215646 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:06.217993 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.219189 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.221593 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.222142 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.222262 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.226739 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:06.226747 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:06.226752 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:06.226801 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:06.230434 (+ 3633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:06.231441 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:06.231447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:06.231451 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:06.231633 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:06.232155 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:06.232161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:06.232164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:06.232568 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:06.232723 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:06.233514 (+ 791us) server_negotiation.cc:299] Negotiation successful
0928 14:51:06.233797 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:51:06.234990 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.212072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:45975 (local address 127.14.241.195:57805)
0928 14:51:06.212928 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:06.212968 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:06.215024 (+ 2056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:06.215974 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:06.215987 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:06.216766 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:06.217717 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.217745 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.219387 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.219393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.221386 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.221406 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.222397 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.222405 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.222735 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.223538 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:06.223562 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:06.226460 (+ 2898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:06.230588 (+ 4128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:06.230601 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:06.230621 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:06.231292 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:06.231771 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:06.231778 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:06.231781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:06.232014 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:06.232720 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:06.232730 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:06.233111 (+ 381us) client_negotiation.cc:769] Sending connection context
0928 14:51:06.233353 (+ 242us) client_negotiation.cc:240] Negotiation successful
0928 14:51:06.233762 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":381,"threads_started":1}
I20260928 14:51:06.257519 23135 heartbeater.cc:503] Master 127.14.241.254:45975 was elected leader, sending a full tablet report...
I20260928 14:51:06.259169 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.243821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:45975 (local address 127.14.241.195:53711)
0928 14:51:06.244571 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:06.244594 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:06.244771 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:06.245121 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:06.245126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:06.245756 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:06.246215 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.246230 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.247698 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.247705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.249470 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.249487 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.250878 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.250885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.251084 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.251764 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:06.251788 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:06.252935 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:06.256306 (+ 3371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:06.256311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:06.256315 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:06.256878 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:06.257332 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:06.257338 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:06.257342 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:06.257523 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:06.258098 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:06.258104 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:06.258279 (+ 175us) client_negotiation.cc:769] Sending connection context
0928 14:51:06.258419 (+ 140us) client_negotiation.cc:240] Negotiation successful
0928 14:51:06.258727 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":368,"threads_started":1}
I20260928 14:51:06.259213 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.243409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:53711 (local address 127.14.241.254:45975)
0928 14:51:06.243578 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:06.243584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:06.244700 (+ 1116us) server_negotiation.cc:407] Connection header received
0928 14:51:06.244898 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:06.244902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:06.244969 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:06.245065 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:06.246423 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.247507 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.249634 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.250295 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.250428 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.253106 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:06.253111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:06.253114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:06.253147 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:06.256163 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:06.257013 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:06.257019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:06.257023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:06.257188 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:06.257642 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:06.257647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:06.257650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:06.257982 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:06.258119 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:06.258574 (+ 455us) server_negotiation.cc:299] Negotiation successful
0928 14:51:06.258845 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:51:06.266273 23278 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:44051
I20260928 14:51:06.266328 23412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:44051 every 8 connection(s)
I20260928 14:51:06.267484 23278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:51:06.268195 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 23278
I20260928 14:51:06.268395 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:51:06.268960 15303 external_mini_cluster.cc:1428] Setting key a7979e4ee03b787b26ebefe3c8ef7faf
I20260928 14:51:06.269728 23413 heartbeater.cc:347] Connected to a master server at 127.14.241.254:45975
I20260928 14:51:06.269974 23413 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:06.270696 23413 heartbeater.cc:511] Master 127.14.241.254:45975 requested a full tablet report, sending...
I20260928 14:51:06.272024 22927 ts_manager.cc:194] Registered new tserver with Master: e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051)
I20260928 14:51:06.272863 22927 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:57805
I20260928 14:51:06.284335 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:51:06 dist-test-slave-vk40 krb5kdc[22882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607066, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:51:06 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:06 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 14:51:06.331753 23023 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:06.321460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46918 (local address 127.14.241.254:45975)
0928 14:51:06.321680 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:06.321687 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:06.321917 (+ 230us) server_negotiation.cc:407] Connection header received
0928 14:51:06.322099 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:06.322103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:06.322169 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:06.322252 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:06.323576 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.324350 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.325556 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.325979 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.326085 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.331440 (+ 5355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46918: BlockingRecv error: recv got EOF from 127.0.0.1:46918 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 28 14:51:06 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607066, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:51:06.347747 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.334960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46926 (local address 127.14.241.254:45975)
0928 14:51:06.335161 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:06.335167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:06.335415 (+ 248us) server_negotiation.cc:407] Connection header received
0928 14:51:06.335589 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:06.335592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:06.335713 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:06.335796 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:06.336944 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.337773 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.339020 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.339459 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.339556 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.343155 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:06.343160 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:06.343163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:06.343200 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:06.345556 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:06.346153 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:06.346157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:06.346159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:06.346271 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:06.346557 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:06.346559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:06.346561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:06.346823 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:06.347051 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:06.347314 (+ 263us) server_negotiation.cc:299] Negotiation successful
0928 14:51:06.347470 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:51:06.351639 22927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:51:06.358520 22927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:51:06.380009 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.371482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59146 (local address 127.14.241.194:43229)
0928 14:51:06.373186 (+ 1704us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:06.373196 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:06.373823 (+ 627us) server_negotiation.cc:407] Connection header received
0928 14:51:06.373992 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:06.374001 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:06.374272 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:06.374472 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:06.375138 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.376108 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.378029 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.378921 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.379090 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.379299 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:06.379558 (+ 259us) server_negotiation.cc:299] Negotiation successful
0928 14:51:06.379722 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1547,"thread_start_us":1413,"threads_started":1}
I20260928 14:51:06.380054 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.370971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:43229 (local address 127.0.0.1:59146)
0928 14:51:06.373698 (+ 2727us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:06.373716 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:06.373843 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:06.374662 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:06.374668 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:06.374692 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:06.374980 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.374988 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.376332 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.376338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.377876 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.377890 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.379118 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.379124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.379344 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.379385 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 14:51:06.379499 (+ 114us) client_negotiation.cc:240] Negotiation successful
0928 14:51:06.379707 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2600,"thread_start_us":544,"threads_started":1}
I20260928 14:51:06.385725 23204 tablet_service.cc:1467] Processing CreateTablet for tablet 86676e70e24e47b1a20e8752340b7413 (DEFAULT_TABLE table=test-table [id=e8d2efd1c97544388be33c85d8bf9e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:06.387172 23204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86676e70e24e47b1a20e8752340b7413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:06.387633 23436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.373305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36774 (local address 127.14.241.193:44941)
0928 14:51:06.374292 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:06.374301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:06.380632 (+ 6331us) server_negotiation.cc:407] Connection header received
0928 14:51:06.380765 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:06.380774 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:06.381002 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:06.381194 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:06.381553 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.382923 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.385096 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.386517 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.386703 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.386883 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:06.387159 (+ 276us) server_negotiation.cc:299] Negotiation successful
0928 14:51:06.387324 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":714,"threads_started":1}
I20260928 14:51:06.387878 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.372817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:44941 (local address 127.0.0.1:36774)
0928 14:51:06.380354 (+ 7537us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:06.380379 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:06.380498 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:06.381196 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:06.381205 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:06.381233 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:06.381418 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.381427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.383127 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.383132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.384922 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.384937 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.386703 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.386709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.386961 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.387005 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 14:51:06.387090 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:51:06.387355 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7358,"thread_start_us":8444,"threads_started":1}
I20260928 14:51:06.390601 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.370768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:44051 (local address 127.0.0.1:60246)
0928 14:51:06.373578 (+ 2810us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:06.373624 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:06.373782 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:06.383029 (+ 9247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:06.383034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:06.383064 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:06.383496 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.383507 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.385399 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.385404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.387376 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.387391 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.389898 (+ 2507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.389902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.390026 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.390049 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 14:51:06.390126 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:51:06.390262 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2492,"thread_start_us":2630,"threads_started":1}
I20260928 14:51:06.390651 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.370992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60246 (local address 127.14.241.195:44051)
0928 14:51:06.382533 (+ 11541us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:06.382539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:06.382567 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:51:06.382639 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:06.382644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:06.382825 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:06.383024 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:06.383704 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.385253 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.387567 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.389473 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.389662 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.389784 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:06.390195 (+ 411us) server_negotiation.cc:299] Negotiation successful
0928 14:51:06.390355 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11396,"thread_start_us":11217,"threads_started":1}
I20260928 14:51:06.392656 23065 tablet_service.cc:1467] Processing CreateTablet for tablet 86676e70e24e47b1a20e8752340b7413 (DEFAULT_TABLE table=test-table [id=e8d2efd1c97544388be33c85d8bf9e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:06.394186 23065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86676e70e24e47b1a20e8752340b7413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:06.395382 23343 tablet_service.cc:1467] Processing CreateTablet for tablet 86676e70e24e47b1a20e8752340b7413 (DEFAULT_TABLE table=test-table [id=e8d2efd1c97544388be33c85d8bf9e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:06.396596 23343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86676e70e24e47b1a20e8752340b7413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:06.402606 23437 tablet_bootstrap.cc:492] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: Bootstrap starting.
I20260928 14:51:06.406785 23437 tablet_bootstrap.cc:654] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:06.407661 23438 tablet_bootstrap.cc:492] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: Bootstrap starting.
I20260928 14:51:06.408116 23437 log.cc:824] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: Log is configured to *not* fsync() on all Append() calls
I20260928 14:51:06.411106 23438 tablet_bootstrap.cc:654] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:06.411135 23437 tablet_bootstrap.cc:492] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: No bootstrap required, opened a new log
I20260928 14:51:06.411432 23437 ts_tablet_manager.cc:1397] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 14:51:06.412328 23438 log.cc:824] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:51:06.412415 23439 tablet_bootstrap.cc:492] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: Bootstrap starting.
I20260928 14:51:06.415513 23438 tablet_bootstrap.cc:492] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: No bootstrap required, opened a new log
I20260928 14:51:06.415727 23439 tablet_bootstrap.cc:654] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:06.415845 23438 ts_tablet_manager.cc:1397] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.007s
I20260928 14:51:06.417001 23439 log.cc:824] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:51:06.419402 23437 raft_consensus.cc:348] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.419757 23437 raft_consensus.cc:374] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:06.419847 23437 raft_consensus.cc:729] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83c56e4461d4027addc640ce7929367, State: Initialized, Role: FOLLOWER
I20260928 14:51:06.420208 23439 tablet_bootstrap.cc:492] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: No bootstrap required, opened a new log
I20260928 14:51:06.420508 23439 ts_tablet_manager.cc:1397] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260928 14:51:06.420599 23437 consensus_queue.cc:260] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.424122 23274 heartbeater.cc:503] Master 127.14.241.254:45975 was elected leader, sending a full tablet report...
I20260928 14:51:06.425949 23438 raft_consensus.cc:348] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.426285 23437 ts_tablet_manager.cc:1428] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260928 14:51:06.426394 23438 raft_consensus.cc:374] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:06.426483 23438 raft_consensus.cc:729] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a93577de632b4d6d919d2c5d9461e6a2, State: Initialized, Role: FOLLOWER
I20260928 14:51:06.427136 23438 consensus_queue.cc:260] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.428915 23438 ts_tablet_manager.cc:1428] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260928 14:51:06.430573 23439 raft_consensus.cc:348] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.431034 23439 raft_consensus.cc:374] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:06.431129 23439 raft_consensus.cc:729] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e457bb98dd478caa3cccb3bfd88a4b, State: Initialized, Role: FOLLOWER
I20260928 14:51:06.431831 23439 consensus_queue.cc:260] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.432961 23413 heartbeater.cc:503] Master 127.14.241.254:45975 was elected leader, sending a full tablet report...
I20260928 14:51:06.433492 23439 ts_tablet_manager.cc:1428] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
W20260928 14:51:06.501003 23136 tablet.cc:2369] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:51:06.510377 23275 tablet.cc:2369] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:51:06.519194 23414 tablet.cc:2369] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:51:06.582067 23443 raft_consensus.cc:482] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:51:06.582398 23443 raft_consensus.cc:504] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.584089 23443 leader_election.cc:288] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051), a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941)
I20260928 14:51:06.590404 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.584556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:54051 (local address 127.14.241.195:44051)
0928 14:51:06.584790 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:06.584797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:06.585273 (+ 476us) server_negotiation.cc:407] Connection header received
0928 14:51:06.585472 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:06.585476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:06.585536 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:06.585610 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:06.586139 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.586897 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.588762 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.589532 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.589703 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.589778 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:06.590033 (+ 255us) server_negotiation.cc:299] Negotiation successful
0928 14:51:06.590151 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260928 14:51:06.590536 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.584511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:44051 (local address 127.14.241.194:54051)
0928 14:51:06.585149 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:06.585166 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:06.585332 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:06.585644 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:06.585649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:06.585682 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:06.586003 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.586012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.587026 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.587031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.588592 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.588605 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.589731 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.589737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.589886 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.589910 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 14:51:06.589990 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 14:51:06.590122 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"spinlock_wait_cycles":5504,"thread_start_us":327,"threads_started":1}
I20260928 14:51:06.590785 23436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.584617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:33495 (local address 127.14.241.193:44941)
0928 14:51:06.584852 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:06.584858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:06.585716 (+ 858us) server_negotiation.cc:407] Connection header received
0928 14:51:06.585897 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:06.585901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:06.585970 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:06.586044 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:06.586560 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.587425 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.589176 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.589946 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.590092 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.590164 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:06.590429 (+ 265us) server_negotiation.cc:299] Negotiation successful
0928 14:51:06.590557 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260928 14:51:06.591094 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.585021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:44941 (local address 127.14.241.194:33495)
0928 14:51:06.585611 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:06.585624 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:06.585765 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:06.586082 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:06.586086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:06.586128 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:06.586416 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.586423 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.587561 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.587565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.589026 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.589038 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.590148 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.590154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.590290 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.590313 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 14:51:06.590394 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 14:51:06.590578 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":31,"thread_start_us":279,"threads_started":1}
I20260928 14:51:06.591583 23363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86676e70e24e47b1a20e8752340b7413" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" is_pre_election: true
I20260928 14:51:06.592095 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86676e70e24e47b1a20e8752340b7413" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93577de632b4d6d919d2c5d9461e6a2" is_pre_election: true
I20260928 14:51:06.592219 23363 raft_consensus.cc:2454] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 0.
I20260928 14:51:06.592538 23087 raft_consensus.cc:2454] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 0.
I20260928 14:51:06.592905 23158 leader_election.cc:302] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d83c56e4461d4027addc640ce7929367, e2e457bb98dd478caa3cccb3bfd88a4b; no voters:
I20260928 14:51:06.593340 23443 raft_consensus.cc:2781] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:51:06.593511 23443 raft_consensus.cc:482] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:51:06.593598 23443 raft_consensus.cc:3037] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:06.596760 23443 raft_consensus.cc:504] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.597465 23443 leader_election.cc:288] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 election: Requested vote from peers e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051), a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941)
I20260928 14:51:06.597968 23363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86676e70e24e47b1a20e8752340b7413" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b"
I20260928 14:51:06.597991 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86676e70e24e47b1a20e8752340b7413" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93577de632b4d6d919d2c5d9461e6a2"
I20260928 14:51:06.598184 23087 raft_consensus.cc:3037] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:06.598187 23363 raft_consensus.cc:3037] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:06.600402 23087 raft_consensus.cc:2454] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 1.
I20260928 14:51:06.600847 23157 leader_election.cc:302] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a93577de632b4d6d919d2c5d9461e6a2, d83c56e4461d4027addc640ce7929367; no voters:
I20260928 14:51:06.601117 23443 raft_consensus.cc:2781] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:06.601495 23363 raft_consensus.cc:2454] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 1.
I20260928 14:51:06.601594 23443 raft_consensus.cc:686] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 1 LEADER]: Becoming Leader. State: Replica: d83c56e4461d4027addc640ce7929367, State: Running, Role: LEADER
I20260928 14:51:06.602155 23443 consensus_queue.cc:237] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.607299 22927 catalog_manager.cc:5495] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 reported cstate change: term changed from 0 to 1, leader changed from <none> to d83c56e4461d4027addc640ce7929367 (127.14.241.194). New cstate: current_term: 1 leader_uuid: "d83c56e4461d4027addc640ce7929367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } health_report { overall_health: UNKNOWN } } }
I20260928 14:51:06.630378 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.625284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.14.241.194:43229)
0928 14:51:06.625453 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:06.625457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:06.625475 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:51:06.625564 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:06.625568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:06.625643 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:06.625777 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:51:06.626196 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.626963 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.628339 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.628788 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.628889 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.629304 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:51:06.629878 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:51:06.629994 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:06.630038 (+ 44us) server_negotiation.cc:299] Negotiation successful
0928 14:51:06.630108 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:51:06.638988 23363 raft_consensus.cc:1261] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Refusing update from remote peer d83c56e4461d4027addc640ce7929367: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:06.639024 23087 raft_consensus.cc:1261] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Refusing update from remote peer d83c56e4461d4027addc640ce7929367: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:06.640020 23448 consensus_queue.cc:1035] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:06.640369 23443 consensus_queue.cc:1035] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:06.652576 23451 mvcc.cc:205] Tried to move back new op lower bound from 7334326544942825472 to 7334326544806354944. Current Snapshot: MvccSnapshot[applied={T|T < 7334326544942825472}]
I20260928 14:51:06.665774 22927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:06.666095 22927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:06.670040 22927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b: Sending DeleteTablet for 3 replicas of tablet 86676e70e24e47b1a20e8752340b7413
I20260928 14:51:06.671182 23343 tablet_service.cc:1514] Processing DeleteTablet for tablet 86676e70e24e47b1a20e8752340b7413 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:06 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:60246
I20260928 14:51:06.671581 23065 tablet_service.cc:1514] Processing DeleteTablet for tablet 86676e70e24e47b1a20e8752340b7413 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:06 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:36774
I20260928 14:51:06.671929 23204 tablet_service.cc:1514] Processing DeleteTablet for tablet 86676e70e24e47b1a20e8752340b7413 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:06 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59146
I20260928 14:51:06.672899 23463 tablet_replica.cc:330] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: stopping tablet replica
I20260928 14:51:06.672669 22927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:51:06.673410 22927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:51:06.673385 23463 raft_consensus.cc:2229] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:51:06.674050 23463 raft_consensus.cc:2258] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:06.676187 23463 ts_tablet_manager.cc:1905] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:06.683645 23461 tablet_replica.cc:330] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: stopping tablet replica
I20260928 14:51:06.683911 23463 ts_tablet_manager.cc:1918] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:06.684096 23463 log.cc:1196] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/86676e70e24e47b1a20e8752340b7413
I20260928 14:51:06.684679 23463 ts_tablet_manager.cc:1939] T 86676e70e24e47b1a20e8752340b7413 P d83c56e4461d4027addc640ce7929367: Deleting consensus metadata
I20260928 14:51:06.684814 23204 tablet_service.cc:1467] Processing CreateTablet for tablet 20299157c978477584adf24220eb45d9 (DEFAULT_TABLE table=test-table [id=791429210f704fd3acc3334ad44c8f78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:06.684922 23343 tablet_service.cc:1467] Processing CreateTablet for tablet 20299157c978477584adf24220eb45d9 (DEFAULT_TABLE table=test-table [id=791429210f704fd3acc3334ad44c8f78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:06.685305 23204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20299157c978477584adf24220eb45d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:06.685554 23343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20299157c978477584adf24220eb45d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:06.685962 23064 tablet_service.cc:1467] Processing CreateTablet for tablet 20299157c978477584adf24220eb45d9 (DEFAULT_TABLE table=test-table [id=791429210f704fd3acc3334ad44c8f78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:06.686539 23064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20299157c978477584adf24220eb45d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:06.690927 23461 raft_consensus.cc:2229] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:06.691519 23461 raft_consensus.cc:2258] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:06.692575 23437 tablet_bootstrap.cc:492] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367: Bootstrap starting.
I20260928 14:51:06.693688 23438 tablet_bootstrap.cc:492] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2: Bootstrap starting.
I20260928 14:51:06.694387 23437 tablet_bootstrap.cc:654] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:06.694860 23461 ts_tablet_manager.cc:1905] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:06.695240 23438 tablet_bootstrap.cc:654] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:06.696233 23439 tablet_bootstrap.cc:492] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b: Bootstrap starting.
I20260928 14:51:06.698050 23439 tablet_bootstrap.cc:654] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:06.701022 22914 catalog_manager.cc:4841] TS d83c56e4461d4027addc640ce7929367 (127.14.241.194:43229): tablet 86676e70e24e47b1a20e8752340b7413 (table test-table [id=e8d2efd1c97544388be33c85d8bf9e46]) successfully deleted
I20260928 14:51:06.706653 23439 tablet_bootstrap.cc:492] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b: No bootstrap required, opened a new log
I20260928 14:51:06.706907 23439 ts_tablet_manager.cc:1397] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.010s
I20260928 14:51:06.707762 23439 raft_consensus.cc:348] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:06.708086 23439 raft_consensus.cc:374] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:06.708238 23439 raft_consensus.cc:729] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e457bb98dd478caa3cccb3bfd88a4b, State: Initialized, Role: FOLLOWER
I20260928 14:51:06.708633 23439 consensus_queue.cc:260] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:06.711613 23439 ts_tablet_manager.cc:1428] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260928 14:51:06.718089 23438 tablet_bootstrap.cc:492] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2: No bootstrap required, opened a new log
I20260928 14:51:06.718263 23438 ts_tablet_manager.cc:1397] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.000s
I20260928 14:51:06.718808 23437 tablet_bootstrap.cc:492] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367: No bootstrap required, opened a new log
I20260928 14:51:06.718968 23437 ts_tablet_manager.cc:1397] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367: Time spent bootstrapping tablet: real 0.026s user 0.018s sys 0.000s
I20260928 14:51:06.719108 23438 raft_consensus.cc:348] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:06.719347 23438 raft_consensus.cc:374] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:06.719447 23438 raft_consensus.cc:729] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a93577de632b4d6d919d2c5d9461e6a2, State: Initialized, Role: FOLLOWER
I20260928 14:51:06.719751 23438 consensus_queue.cc:260] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:06.719787 23437 raft_consensus.cc:348] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:06.720028 23437 raft_consensus.cc:374] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:06.720127 23437 raft_consensus.cc:729] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83c56e4461d4027addc640ce7929367, State: Initialized, Role: FOLLOWER
I20260928 14:51:06.720463 23437 consensus_queue.cc:260] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:06.722617 23461 ts_tablet_manager.cc:1918] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:06.722658 23438 ts_tablet_manager.cc:1428] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260928 14:51:06.722870 23461 log.cc:1196] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/86676e70e24e47b1a20e8752340b7413
I20260928 14:51:06.723213 23437 ts_tablet_manager.cc:1428] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260928 14:51:06.723505 23461 ts_tablet_manager.cc:1939] T 86676e70e24e47b1a20e8752340b7413 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting consensus metadata
I20260928 14:51:06.726135 22915 catalog_manager.cc:4841] TS e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051): tablet 86676e70e24e47b1a20e8752340b7413 (table test-table [id=e8d2efd1c97544388be33c85d8bf9e46]) successfully deleted
I20260928 14:51:06.726410 23462 tablet_replica.cc:330] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: stopping tablet replica
I20260928 14:51:06.726775 23462 raft_consensus.cc:2229] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:06.727166 23462 raft_consensus.cc:2258] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:06.728842 23462 ts_tablet_manager.cc:1905] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:06.734009 23462 ts_tablet_manager.cc:1918] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:06.734157 23462 log.cc:1196] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/86676e70e24e47b1a20e8752340b7413
I20260928 14:51:06.734603 23462 ts_tablet_manager.cc:1939] T 86676e70e24e47b1a20e8752340b7413 P a93577de632b4d6d919d2c5d9461e6a2: Deleting consensus metadata
I20260928 14:51:06.736145 22914 catalog_manager.cc:4841] TS a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941): tablet 86676e70e24e47b1a20e8752340b7413 (table test-table [id=e8d2efd1c97544388be33c85d8bf9e46]) successfully deleted
I20260928 14:51:06.851989 23443 raft_consensus.cc:482] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:51:06.852193 23443 raft_consensus.cc:504] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:06.852859 23443 leader_election.cc:288] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941), e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051)
I20260928 14:51:06.853426 23363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20299157c978477584adf24220eb45d9" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" is_pre_election: true
I20260928 14:51:06.853462 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20299157c978477584adf24220eb45d9" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93577de632b4d6d919d2c5d9461e6a2" is_pre_election: true
I20260928 14:51:06.853703 23087 raft_consensus.cc:2454] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 0.
I20260928 14:51:06.853689 23363 raft_consensus.cc:2454] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 0.
I20260928 14:51:06.854116 23157 leader_election.cc:302] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a93577de632b4d6d919d2c5d9461e6a2, d83c56e4461d4027addc640ce7929367; no voters:
I20260928 14:51:06.854383 23443 raft_consensus.cc:2781] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:51:06.854511 23443 raft_consensus.cc:482] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:51:06.854619 23443 raft_consensus.cc:3037] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:06.857740 23443 raft_consensus.cc:504] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:06.858407 23443 leader_election.cc:288] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 election: Requested vote from peers a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941), e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051)
I20260928 14:51:06.858978 23363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20299157c978477584adf24220eb45d9" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b"
I20260928 14:51:06.859194 23363 raft_consensus.cc:3037] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:06.859189 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20299157c978477584adf24220eb45d9" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93577de632b4d6d919d2c5d9461e6a2"
I20260928 14:51:06.859402 23087 raft_consensus.cc:3037] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:06.862083 23363 raft_consensus.cc:2454] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 1.
I20260928 14:51:06.862172 23087 raft_consensus.cc:2454] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 1.
I20260928 14:51:06.862643 23157 leader_election.cc:302] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a93577de632b4d6d919d2c5d9461e6a2, d83c56e4461d4027addc640ce7929367, e2e457bb98dd478caa3cccb3bfd88a4b; no voters:
I20260928 14:51:06.862990 23443 raft_consensus.cc:2781] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:06.863142 23443 raft_consensus.cc:686] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 1 LEADER]: Becoming Leader. State: Replica: d83c56e4461d4027addc640ce7929367, State: Running, Role: LEADER
I20260928 14:51:06.863407 23443 consensus_queue.cc:237] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:06.865936 22927 catalog_manager.cc:5495] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 reported cstate change: term changed from 0 to 1, leader changed from <none> to d83c56e4461d4027addc640ce7929367 (127.14.241.194). New cstate: current_term: 1 leader_uuid: "d83c56e4461d4027addc640ce7929367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } health_report { overall_health: UNKNOWN } } }
Sep 28 14:51:06 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607064, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.254@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:51:06.949213 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.934966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46940 (local address 127.14.241.254:45975)
0928 14:51:06.935623 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:06.935629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:06.935861 (+ 232us) server_negotiation.cc:407] Connection header received
0928 14:51:06.936039 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:06.936044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:06.936101 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:06.936187 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:06.937525 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.938405 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.939680 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.940073 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.940180 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.943764 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:06.943769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:06.943772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:06.943806 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:06.946414 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:06.947150 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:06.947154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:06.947156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:06.947273 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:06.947623 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:06.947628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:06.947631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:06.947926 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:06.948078 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:06.948556 (+ 478us) server_negotiation.cc:299] Negotiation successful
0928 14:51:06.948817 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":492,"thread_start_us":307,"threads_started":1}
I20260928 14:51:06.949334 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:06.934997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:45975 (local address 127.0.0.1:46940)
0928 14:51:06.935759 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:06.935775 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:06.935932 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:06.936237 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:06.936242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:06.937100 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:06.937358 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.937369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.938531 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.938536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.939556 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:06.939569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:06.940230 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:06.940236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:06.940409 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:06.941051 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:06.941068 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:06.943522 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:06.946554 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:06.946570 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:06.946578 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:06.947023 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:06.947373 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:06.947378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:06.947382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:06.947522 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:06.948054 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:06.948063 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:06.948234 (+ 171us) client_negotiation.cc:769] Sending connection context
0928 14:51:06.948438 (+ 204us) client_negotiation.cc:240] Negotiation successful
0928 14:51:06.948941 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":170,"thread_start_us":482,"threads_started":1}
I20260928 14:51:06.955324 22927 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:46940:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 14:51:06.956347 22927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:51:06.963346 23343 tablet_service.cc:1467] Processing CreateTablet for tablet 8c182c137d9540b4800612c47d1d13e0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c8ffcbccbaf14299a42ee76cd179ca0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:51:06.963351 23204 tablet_service.cc:1467] Processing CreateTablet for tablet 8c182c137d9540b4800612c47d1d13e0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c8ffcbccbaf14299a42ee76cd179ca0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:51:06.963893 23343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c182c137d9540b4800612c47d1d13e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:06.963954 23204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c182c137d9540b4800612c47d1d13e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:06.969455 23065 tablet_service.cc:1467] Processing CreateTablet for tablet 8c182c137d9540b4800612c47d1d13e0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c8ffcbccbaf14299a42ee76cd179ca0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:51:06.969990 23065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c182c137d9540b4800612c47d1d13e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:06.971532 23437 tablet_bootstrap.cc:492] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: Bootstrap starting.
I20260928 14:51:06.972527 23439 tablet_bootstrap.cc:492] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b: Bootstrap starting.
I20260928 14:51:06.973578 23437 tablet_bootstrap.cc:654] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:06.974380 23439 tablet_bootstrap.cc:654] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:06.977125 23438 tablet_bootstrap.cc:492] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2: Bootstrap starting.
I20260928 14:51:06.978981 23438 tablet_bootstrap.cc:654] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:06.985741 23437 tablet_bootstrap.cc:492] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: No bootstrap required, opened a new log
I20260928 14:51:06.985956 23437 ts_tablet_manager.cc:1397] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260928 14:51:06.986871 23437 raft_consensus.cc:348] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.987151 23437 raft_consensus.cc:374] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:06.987221 23437 raft_consensus.cc:729] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83c56e4461d4027addc640ce7929367, State: Initialized, Role: FOLLOWER
I20260928 14:51:06.987507 23437 consensus_queue.cc:260] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.988013 23443 tablet_replica.cc:439] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } }
I20260928 14:51:06.988369 23443 tablet_replica.cc:442] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:51:06.989336 23439 tablet_bootstrap.cc:492] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b: No bootstrap required, opened a new log
I20260928 14:51:06.989377 23438 tablet_bootstrap.cc:492] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2: No bootstrap required, opened a new log
I20260928 14:51:06.989532 23438 ts_tablet_manager.cc:1397] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260928 14:51:06.989574 23439 ts_tablet_manager.cc:1397] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.000s
I20260928 14:51:06.990280 23438 raft_consensus.cc:348] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.990540 23438 raft_consensus.cc:374] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:06.990617 23438 raft_consensus.cc:729] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a93577de632b4d6d919d2c5d9461e6a2, State: Initialized, Role: FOLLOWER
I20260928 14:51:06.990411 23439 raft_consensus.cc:348] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.990939 23439 raft_consensus.cc:374] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:06.990854 23438 consensus_queue.cc:260] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.991097 23439 raft_consensus.cc:729] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e457bb98dd478caa3cccb3bfd88a4b, State: Initialized, Role: FOLLOWER
I20260928 14:51:06.991379 23437 ts_tablet_manager.cc:1428] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260928 14:51:06.991282 23444 tablet_replica.cc:439] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } }
I20260928 14:51:06.991407 23439 consensus_queue.cc:260] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:06.991638 23444 tablet_replica.cc:442] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:51:06.991914 23445 tablet_replica.cc:439] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } }
I20260928 14:51:06.992203 23445 tablet_replica.cc:442] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:51:06.993749 23438 ts_tablet_manager.cc:1428] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260928 14:51:06.993803 23439 ts_tablet_manager.cc:1428] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260928 14:51:07.165030 23443 raft_consensus.cc:482] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:51:07.165243 23443 raft_consensus.cc:504] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:07.165877 23443 leader_election.cc:288] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051), a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941)
I20260928 14:51:07.166411 23363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c182c137d9540b4800612c47d1d13e0" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" is_pre_election: true
I20260928 14:51:07.166435 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c182c137d9540b4800612c47d1d13e0" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93577de632b4d6d919d2c5d9461e6a2" is_pre_election: true
I20260928 14:51:07.166692 23363 raft_consensus.cc:2454] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 0.
I20260928 14:51:07.166692 23087 raft_consensus.cc:2454] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 0.
I20260928 14:51:07.167090 23158 leader_election.cc:302] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d83c56e4461d4027addc640ce7929367, e2e457bb98dd478caa3cccb3bfd88a4b; no voters:
I20260928 14:51:07.167382 23443 raft_consensus.cc:2781] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:51:07.167487 23443 raft_consensus.cc:482] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:51:07.167591 23443 raft_consensus.cc:3037] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:07.169696 23443 raft_consensus.cc:504] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:07.170295 23443 leader_election.cc:288] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 election: Requested vote from peers e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051), a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941)
I20260928 14:51:07.170840 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c182c137d9540b4800612c47d1d13e0" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93577de632b4d6d919d2c5d9461e6a2"
I20260928 14:51:07.170847 23363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c182c137d9540b4800612c47d1d13e0" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b"
I20260928 14:51:07.171087 23363 raft_consensus.cc:3037] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:07.171087 23087 raft_consensus.cc:3037] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:07.174017 23087 raft_consensus.cc:2454] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 1.
I20260928 14:51:07.174017 23363 raft_consensus.cc:2454] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 1.
I20260928 14:51:07.174625 23158 leader_election.cc:302] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a93577de632b4d6d919d2c5d9461e6a2, d83c56e4461d4027addc640ce7929367, e2e457bb98dd478caa3cccb3bfd88a4b; no voters:
I20260928 14:51:07.174995 23443 raft_consensus.cc:2781] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:07.175164 23443 raft_consensus.cc:686] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [term 1 LEADER]: Becoming Leader. State: Replica: d83c56e4461d4027addc640ce7929367, State: Running, Role: LEADER
I20260928 14:51:07.175401 23443 consensus_queue.cc:237] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } }
I20260928 14:51:07.176925 23448 tablet_replica.cc:439] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: TxnStatusTablet state changed. Reason: New leader d83c56e4461d4027addc640ce7929367. Latest consensus state: current_term: 1 leader_uuid: "d83c56e4461d4027addc640ce7929367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } }
I20260928 14:51:07.177217 23448 tablet_replica.cc:442] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:51:07.177819 23479 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 14:51:07.178045 23479 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 14:51:07.178700 22925 catalog_manager.cc:5495] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 reported cstate change: term changed from 0 to 1, leader changed from <none> to d83c56e4461d4027addc640ce7929367 (127.14.241.194). New cstate: current_term: 1 leader_uuid: "d83c56e4461d4027addc640ce7929367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } health_report { overall_health: UNKNOWN } } }
I20260928 14:51:07.226326 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.221041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59158 (local address 127.14.241.194:43229)
0928 14:51:07.221852 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:07.221857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:07.221873 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:51:07.221928 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:07.221932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:07.221994 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:07.222076 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:51:07.222480 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.223272 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.224593 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.225022 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.225129 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.225548 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:51:07.225663 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:51:07.225751 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:07.225879 (+ 128us) server_negotiation.cc:299] Negotiation successful
0928 14:51:07.225954 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":542,"threads_started":1}
I20260928 14:51:07.226323 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.221032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:43229 (local address 127.0.0.1:59158)
0928 14:51:07.221240 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:07.221260 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:07.221373 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:07.222134 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:07.222138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:07.222151 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:51:07.222318 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.222327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.223400 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.223405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.224451 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.224464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.225163 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.225170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.225322 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.225381 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:51:07.225755 (+ 374us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:51:07.225777 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 14:51:07.225844 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 14:51:07.226020 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 14:51:07.230355 23087 raft_consensus.cc:1261] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Refusing update from remote peer d83c56e4461d4027addc640ce7929367: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:07.230405 23363 raft_consensus.cc:1261] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Refusing update from remote peer d83c56e4461d4027addc640ce7929367: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:07.230927 23448 consensus_queue.cc:1035] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:07.231235 23443 consensus_queue.cc:1035] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:07.233853 23444 tablet_replica.cc:439] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2: TxnStatusTablet state changed. Reason: New leader d83c56e4461d4027addc640ce7929367. Latest consensus state: current_term: 1 leader_uuid: "d83c56e4461d4027addc640ce7929367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } }
I20260928 14:51:07.234108 23444 tablet_replica.cc:442] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:51:07.237390 23483 mvcc.cc:205] Tried to move back new op lower bound from 7334326547369590784 to 7334326547152510976. Current Snapshot: MvccSnapshot[applied={T|T < 7334326547369590784}]
I20260928 14:51:07.241575 23444 tablet_replica.cc:439] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d83c56e4461d4027addc640ce7929367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } }
I20260928 14:51:07.241803 23444 tablet_replica.cc:442] T 8c182c137d9540b4800612c47d1d13e0 P a93577de632b4d6d919d2c5d9461e6a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:51:07.244976 23448 tablet_replica.cc:439] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d83c56e4461d4027addc640ce7929367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } }
I20260928 14:51:07.245260 23448 tablet_replica.cc:442] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:51:07.249671 23445 tablet_replica.cc:439] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b: TxnStatusTablet state changed. Reason: New leader d83c56e4461d4027addc640ce7929367. Latest consensus state: current_term: 1 leader_uuid: "d83c56e4461d4027addc640ce7929367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } }
I20260928 14:51:07.249963 23445 tablet_replica.cc:442] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:51:07.255030 23443 tablet_replica.cc:439] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d83c56e4461d4027addc640ce7929367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } }
I20260928 14:51:07.255374 23443 tablet_replica.cc:442] T 8c182c137d9540b4800612c47d1d13e0 P d83c56e4461d4027addc640ce7929367: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:51:07.257800 23484 mvcc.cc:205] Tried to move back new op lower bound from 7334326547369590784 to 7334326547152510976. Current Snapshot: MvccSnapshot[applied={T|T < 7334326547369590784}]
I20260928 14:51:07.259352 23445 tablet_replica.cc:439] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d83c56e4461d4027addc640ce7929367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } } peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } }
I20260928 14:51:07.259594 23445 tablet_replica.cc:442] T 8c182c137d9540b4800612c47d1d13e0 P e2e457bb98dd478caa3cccb3bfd88a4b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:51:07.261309 23448 consensus_queue.cc:1035] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:07.281791 23448 consensus_queue.cc:1035] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:07.301066 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.295618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:36161 (local address 127.14.241.194:43229)
0928 14:51:07.295804 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:07.295811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:07.296439 (+ 628us) server_negotiation.cc:407] Connection header received
0928 14:51:07.296619 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:07.296624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:07.296685 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:07.296752 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:51:07.297269 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.298005 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.299377 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.299815 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.299924 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.300355 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:51:07.300463 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:51:07.300536 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:07.300672 (+ 136us) server_negotiation.cc:299] Negotiation successful
0928 14:51:07.300748 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":57}
I20260928 14:51:07.301172 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.295684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:43229 (local address 127.14.241.194:36161)
0928 14:51:07.296350 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:07.296369 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:07.296508 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:07.296786 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:07.296790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:07.296800 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:51:07.297136 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.297146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.298117 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.298122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.299246 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.299259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.299980 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.299986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.300112 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.300167 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:51:07.300548 (+ 381us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:51:07.300577 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 14:51:07.300642 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:51:07.300795 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":23,"thread_start_us":341,"threads_started":1}
I20260928 14:51:07.362159 22927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:07.362368 22927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:07.365278 22927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b: Sending DeleteTablet for 3 replicas of tablet 20299157c978477584adf24220eb45d9
I20260928 14:51:07.366048 23065 tablet_service.cc:1514] Processing DeleteTablet for tablet 20299157c978477584adf24220eb45d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:07 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:36774
I20260928 14:51:07.366081 23203 tablet_service.cc:1514] Processing DeleteTablet for tablet 20299157c978477584adf24220eb45d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:07 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59146
I20260928 14:51:07.366174 23343 tablet_service.cc:1514] Processing DeleteTablet for tablet 20299157c978477584adf24220eb45d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:07 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:60246
I20260928 14:51:07.378480 23497 tablet_replica.cc:330] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2: stopping tablet replica
I20260928 14:51:07.378891 23497 raft_consensus.cc:2229] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:07.379268 23497 raft_consensus.cc:2258] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:07.380645 23497 ts_tablet_manager.cc:1905] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:07.381357 23498 tablet_replica.cc:330] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367: stopping tablet replica
I20260928 14:51:07.381577 23498 raft_consensus.cc:2229] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:51:07.382025 23498 raft_consensus.cc:2258] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:07.383143 23498 ts_tablet_manager.cc:1905] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:07.387965 23497 ts_tablet_manager.cc:1918] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:51:07.388178 23497 log.cc:1196] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/20299157c978477584adf24220eb45d9
I20260928 14:51:07.388733 23497 ts_tablet_manager.cc:1939] T 20299157c978477584adf24220eb45d9 P a93577de632b4d6d919d2c5d9461e6a2: Deleting consensus metadata
I20260928 14:51:07.389951 22914 catalog_manager.cc:4841] TS a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941): tablet 20299157c978477584adf24220eb45d9 (table test-table [id=791429210f704fd3acc3334ad44c8f78]) successfully deleted
I20260928 14:51:07.390523 23498 ts_tablet_manager.cc:1918] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:51:07.390651 23498 log.cc:1196] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/20299157c978477584adf24220eb45d9
I20260928 14:51:07.391034 23498 ts_tablet_manager.cc:1939] T 20299157c978477584adf24220eb45d9 P d83c56e4461d4027addc640ce7929367: Deleting consensus metadata
I20260928 14:51:07.391884 23499 tablet_replica.cc:330] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b: stopping tablet replica
I20260928 14:51:07.391980 22914 catalog_manager.cc:4841] TS d83c56e4461d4027addc640ce7929367 (127.14.241.194:43229): tablet 20299157c978477584adf24220eb45d9 (table test-table [id=791429210f704fd3acc3334ad44c8f78]) successfully deleted
I20260928 14:51:07.392078 23499 raft_consensus.cc:2229] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:07.392429 23499 raft_consensus.cc:2258] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:07.393414 23499 ts_tablet_manager.cc:1905] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:07.400499 23499 ts_tablet_manager.cc:1918] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:51:07.400643 23499 log.cc:1196] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/20299157c978477584adf24220eb45d9
I20260928 14:51:07.401057 23499 ts_tablet_manager.cc:1939] T 20299157c978477584adf24220eb45d9 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting consensus metadata
I20260928 14:51:07.402205 22915 catalog_manager.cc:4841] TS e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051): tablet 20299157c978477584adf24220eb45d9 (table test-table [id=791429210f704fd3acc3334ad44c8f78]) successfully deleted
Sep 28 14:51:07 dist-test-slave-vk40 krb5kdc[22882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:51:07 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:07 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 14:51:07.432575 23473 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:07.422468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46956 (local address 127.14.241.254:45975)
0928 14:51:07.422658 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:07.422664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:07.422962 (+ 298us) server_negotiation.cc:407] Connection header received
0928 14:51:07.423122 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:07.423127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:07.423206 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:07.423321 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:07.424567 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.425683 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.426930 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.427597 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.427721 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.432317 (+ 4596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46956: BlockingRecv error: recv got EOF from 127.0.0.1:46956 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 28 14:51:07 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM
I20260928 14:51:07.450255 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.435075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46972 (local address 127.14.241.254:45975)
0928 14:51:07.435266 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:07.435271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:07.435491 (+ 220us) server_negotiation.cc:407] Connection header received
0928 14:51:07.435621 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:07.435627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:07.435708 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:07.435821 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:07.437042 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.438219 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.439484 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.440164 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.440323 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.444147 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:07.444154 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:07.444160 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:07.444206 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:07.447405 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:07.448141 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:07.448146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:07.448149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:07.448341 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:07.448730 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:07.448743 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:07.448746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:07.449137 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:07.449423 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:07.449630 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 14:51:07.449913 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 14:51:07.453773 22927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:51:07.454409 22927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:51:07.461009 23203 tablet_service.cc:1467] Processing CreateTablet for tablet 2b71a6eda3ae4298bdf11521d1ebf478 (DEFAULT_TABLE table=test-table [id=a74e428ccd554797b893b255492f2383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:07.461509 23203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b71a6eda3ae4298bdf11521d1ebf478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:07.464757 23065 tablet_service.cc:1467] Processing CreateTablet for tablet 2b71a6eda3ae4298bdf11521d1ebf478 (DEFAULT_TABLE table=test-table [id=a74e428ccd554797b893b255492f2383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:07.465193 23065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b71a6eda3ae4298bdf11521d1ebf478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:07.466709 23437 tablet_bootstrap.cc:492] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367: Bootstrap starting.
I20260928 14:51:07.466707 23343 tablet_service.cc:1467] Processing CreateTablet for tablet 2b71a6eda3ae4298bdf11521d1ebf478 (DEFAULT_TABLE table=test-table [id=a74e428ccd554797b893b255492f2383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:07.467139 23343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b71a6eda3ae4298bdf11521d1ebf478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:07.468453 23437 tablet_bootstrap.cc:654] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:07.471868 23438 tablet_bootstrap.cc:492] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2: Bootstrap starting.
I20260928 14:51:07.473484 23438 tablet_bootstrap.cc:654] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:07.475422 23439 tablet_bootstrap.cc:492] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b: Bootstrap starting.
I20260928 14:51:07.477066 23439 tablet_bootstrap.cc:654] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:07.483480 23437 tablet_bootstrap.cc:492] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367: No bootstrap required, opened a new log
I20260928 14:51:07.483743 23437 ts_tablet_manager.cc:1397] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.005s
I20260928 14:51:07.484036 23438 tablet_bootstrap.cc:492] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2: No bootstrap required, opened a new log
I20260928 14:51:07.484192 23438 ts_tablet_manager.cc:1397] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.009s
I20260928 14:51:07.485020 23438 raft_consensus.cc:348] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.485302 23438 raft_consensus.cc:374] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:07.485406 23438 raft_consensus.cc:729] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a93577de632b4d6d919d2c5d9461e6a2, State: Initialized, Role: FOLLOWER
I20260928 14:51:07.485663 23438 consensus_queue.cc:260] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.486306 23437 raft_consensus.cc:348] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.486641 23437 raft_consensus.cc:374] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:07.486819 23437 raft_consensus.cc:729] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83c56e4461d4027addc640ce7929367, State: Initialized, Role: FOLLOWER
I20260928 14:51:07.486871 23438 ts_tablet_manager.cc:1428] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:51:07.487135 23437 consensus_queue.cc:260] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.488665 23437 ts_tablet_manager.cc:1428] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260928 14:51:07.489758 23439 tablet_bootstrap.cc:492] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b: No bootstrap required, opened a new log
I20260928 14:51:07.489913 23439 ts_tablet_manager.cc:1397] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.003s
I20260928 14:51:07.490638 23439 raft_consensus.cc:348] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.490957 23439 raft_consensus.cc:374] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:07.491065 23439 raft_consensus.cc:729] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e457bb98dd478caa3cccb3bfd88a4b, State: Initialized, Role: FOLLOWER
I20260928 14:51:07.491288 23439 consensus_queue.cc:260] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.491954 23439 ts_tablet_manager.cc:1428] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260928 14:51:07.492568 23448 raft_consensus.cc:482] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:51:07.492722 23448 raft_consensus.cc:504] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.493402 23448 leader_election.cc:288] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941), e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051)
I20260928 14:51:07.493742 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b71a6eda3ae4298bdf11521d1ebf478" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93577de632b4d6d919d2c5d9461e6a2" is_pre_election: true
I20260928 14:51:07.493989 23087 raft_consensus.cc:2454] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 0.
I20260928 14:51:07.494120 23363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b71a6eda3ae4298bdf11521d1ebf478" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" is_pre_election: true
I20260928 14:51:07.494344 23363 raft_consensus.cc:2454] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 0.
I20260928 14:51:07.494397 23157 leader_election.cc:302] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a93577de632b4d6d919d2c5d9461e6a2, d83c56e4461d4027addc640ce7929367; no voters:
I20260928 14:51:07.494669 23448 raft_consensus.cc:2781] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:51:07.494828 23448 raft_consensus.cc:482] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:51:07.494935 23448 raft_consensus.cc:3037] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:07.497562 23448 raft_consensus.cc:504] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.498095 23448 leader_election.cc:288] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 election: Requested vote from peers a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941), e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051)
I20260928 14:51:07.498453 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b71a6eda3ae4298bdf11521d1ebf478" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93577de632b4d6d919d2c5d9461e6a2"
I20260928 14:51:07.498543 23363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b71a6eda3ae4298bdf11521d1ebf478" candidate_uuid: "d83c56e4461d4027addc640ce7929367" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b"
I20260928 14:51:07.498661 23087 raft_consensus.cc:3037] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:07.498749 23363 raft_consensus.cc:3037] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:07.501457 23087 raft_consensus.cc:2454] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 1.
I20260928 14:51:07.501456 23363 raft_consensus.cc:2454] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83c56e4461d4027addc640ce7929367 in term 1.
I20260928 14:51:07.501922 23157 leader_election.cc:302] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a93577de632b4d6d919d2c5d9461e6a2, d83c56e4461d4027addc640ce7929367; no voters:
I20260928 14:51:07.502228 23448 raft_consensus.cc:2781] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:07.502368 23448 raft_consensus.cc:686] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 1 LEADER]: Becoming Leader. State: Replica: d83c56e4461d4027addc640ce7929367, State: Running, Role: LEADER
I20260928 14:51:07.502655 23448 consensus_queue.cc:237] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.505309 22928 catalog_manager.cc:5495] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 reported cstate change: term changed from 0 to 1, leader changed from <none> to d83c56e4461d4027addc640ce7929367 (127.14.241.194). New cstate: current_term: 1 leader_uuid: "d83c56e4461d4027addc640ce7929367" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } health_report { overall_health: UNKNOWN } } }
I20260928 14:51:07.524274 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.518593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59160 (local address 127.14.241.194:43229)
0928 14:51:07.519110 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:07.519114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:07.519131 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:51:07.519198 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:07.519202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:07.519261 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:07.519324 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:51:07.519780 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.520711 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.522517 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.522968 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.523059 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.523551 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:51:07.523657 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:51:07.523722 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:07.523895 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 14:51:07.523960 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392}
I20260928 14:51:07.527225 23087 raft_consensus.cc:1261] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Refusing update from remote peer d83c56e4461d4027addc640ce7929367: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:07.527225 23363 raft_consensus.cc:1261] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Refusing update from remote peer d83c56e4461d4027addc640ce7929367: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:07.527884 23448 consensus_queue.cc:1035] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:07.528119 23443 consensus_queue.cc:1035] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:07.540915 22928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:07.541162 22928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:07.545471 22928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b: Sending DeleteTablet for 3 replicas of tablet 2b71a6eda3ae4298bdf11521d1ebf478
I20260928 14:51:07.546190 23065 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b71a6eda3ae4298bdf11521d1ebf478 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:07 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:36774
I20260928 14:51:07.546514 23203 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b71a6eda3ae4298bdf11521d1ebf478 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:07 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59146
I20260928 14:51:07.546787 23497 tablet_replica.cc:330] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2: stopping tablet replica
I20260928 14:51:07.546788 23498 tablet_replica.cc:330] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367: stopping tablet replica
I20260928 14:51:07.546986 23497 raft_consensus.cc:2229] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:07.546993 23498 raft_consensus.cc:2229] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:51:07.547187 23497 raft_consensus.cc:2258] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:07.547310 23498 raft_consensus.cc:2258] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:07.548494 23498 ts_tablet_manager.cc:1905] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:07.548887 23203 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b71a6eda3ae4298bdf11521d1ebf478 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:07 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59146
I20260928 14:51:07.550215 23497 ts_tablet_manager.cc:1905] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:07.553785 23498 ts_tablet_manager.cc:1918] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:07.553951 23498 log.cc:1196] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/2b71a6eda3ae4298bdf11521d1ebf478
I20260928 14:51:07.554301 23498 ts_tablet_manager.cc:1939] T 2b71a6eda3ae4298bdf11521d1ebf478 P d83c56e4461d4027addc640ce7929367: Deleting consensus metadata
I20260928 14:51:07.555218 22914 catalog_manager.cc:4841] TS d83c56e4461d4027addc640ce7929367 (127.14.241.194:43229): tablet 2b71a6eda3ae4298bdf11521d1ebf478 (table test-table [id=a74e428ccd554797b893b255492f2383]) successfully deleted
W20260928 14:51:07.555687 22914 catalog_manager.cc:4808] TS d83c56e4461d4027addc640ce7929367 (127.14.241.194:43229): delete failed for tablet 2b71a6eda3ae4298bdf11521d1ebf478 because the tablet was not found. No further retry: Not found: Tablet not found: 2b71a6eda3ae4298bdf11521d1ebf478
I20260928 14:51:07.556071 23343 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b71a6eda3ae4298bdf11521d1ebf478 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:07 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:60246
I20260928 14:51:07.556367 23499 tablet_replica.cc:330] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b: stopping tablet replica
I20260928 14:51:07.556584 23499 raft_consensus.cc:2229] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:07.556850 23499 raft_consensus.cc:2258] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:07.557510 22928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 14:51:07.557770 23497 ts_tablet_manager.cc:1918] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:07.557924 23497 log.cc:1196] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/2b71a6eda3ae4298bdf11521d1ebf478
W20260928 14:51:07.558128 22928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:51:07.558323 23497 ts_tablet_manager.cc:1939] T 2b71a6eda3ae4298bdf11521d1ebf478 P a93577de632b4d6d919d2c5d9461e6a2: Deleting consensus metadata
I20260928 14:51:07.559808 23499 ts_tablet_manager.cc:1905] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:07.560112 22914 catalog_manager.cc:4841] TS a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941): tablet 2b71a6eda3ae4298bdf11521d1ebf478 (table test-table [id=a74e428ccd554797b893b255492f2383]) successfully deleted
I20260928 14:51:07.566188 23499 ts_tablet_manager.cc:1918] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:07.566351 23499 log.cc:1196] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/2b71a6eda3ae4298bdf11521d1ebf478
I20260928 14:51:07.566766 23499 ts_tablet_manager.cc:1939] T 2b71a6eda3ae4298bdf11521d1ebf478 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting consensus metadata
I20260928 14:51:07.567775 22915 catalog_manager.cc:4841] TS e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051): tablet 2b71a6eda3ae4298bdf11521d1ebf478 (table test-table [id=a74e428ccd554797b893b255492f2383]) successfully deleted
I20260928 14:51:07.569402 23065 tablet_service.cc:1467] Processing CreateTablet for tablet a4aca40cc49f4250a1ad122412fd8eb7 (DEFAULT_TABLE table=test-table [id=3c8fa67d09044554a9bf8197ffa23227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:07.569401 23343 tablet_service.cc:1467] Processing CreateTablet for tablet a4aca40cc49f4250a1ad122412fd8eb7 (DEFAULT_TABLE table=test-table [id=3c8fa67d09044554a9bf8197ffa23227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:07.569816 23343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4aca40cc49f4250a1ad122412fd8eb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:07.569816 23065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4aca40cc49f4250a1ad122412fd8eb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:07.575340 23203 tablet_service.cc:1467] Processing CreateTablet for tablet a4aca40cc49f4250a1ad122412fd8eb7 (DEFAULT_TABLE table=test-table [id=3c8fa67d09044554a9bf8197ffa23227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:07.575847 23438 tablet_bootstrap.cc:492] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2: Bootstrap starting.
I20260928 14:51:07.575834 23203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4aca40cc49f4250a1ad122412fd8eb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:07.576620 23439 tablet_bootstrap.cc:492] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b: Bootstrap starting.
I20260928 14:51:07.577440 23438 tablet_bootstrap.cc:654] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:07.578233 23439 tablet_bootstrap.cc:654] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:07.583477 23437 tablet_bootstrap.cc:492] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367: Bootstrap starting.
I20260928 14:51:07.585146 23437 tablet_bootstrap.cc:654] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:07.587117 23438 tablet_bootstrap.cc:492] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2: No bootstrap required, opened a new log
I20260928 14:51:07.587122 23437 tablet_bootstrap.cc:492] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367: No bootstrap required, opened a new log
I20260928 14:51:07.587292 23437 ts_tablet_manager.cc:1397] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260928 14:51:07.587283 23438 ts_tablet_manager.cc:1397] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260928 14:51:07.588092 23438 raft_consensus.cc:348] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.588382 23438 raft_consensus.cc:374] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:07.588469 23438 raft_consensus.cc:729] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a93577de632b4d6d919d2c5d9461e6a2, State: Initialized, Role: FOLLOWER
I20260928 14:51:07.588718 23438 consensus_queue.cc:260] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.588903 23437 raft_consensus.cc:348] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.589154 23437 raft_consensus.cc:374] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:07.589233 23437 raft_consensus.cc:729] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83c56e4461d4027addc640ce7929367, State: Initialized, Role: FOLLOWER
I20260928 14:51:07.589260 23438 ts_tablet_manager.cc:1428] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:51:07.589524 23437 consensus_queue.cc:260] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.590281 23437 ts_tablet_manager.cc:1428] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:51:07.591866 23439 tablet_bootstrap.cc:492] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b: No bootstrap required, opened a new log
I20260928 14:51:07.592103 23439 ts_tablet_manager.cc:1397] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.011s
I20260928 14:51:07.596621 23439 raft_consensus.cc:348] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.596889 23439 raft_consensus.cc:374] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:07.596959 23439 raft_consensus.cc:729] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e457bb98dd478caa3cccb3bfd88a4b, State: Initialized, Role: FOLLOWER
I20260928 14:51:07.597188 23439 consensus_queue.cc:260] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.598342 23439 ts_tablet_manager.cc:1428] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260928 14:51:07.720207 23444 raft_consensus.cc:482] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:51:07.720459 23444 raft_consensus.cc:504] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.721886 23444 leader_election.cc:288] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d83c56e4461d4027addc640ce7929367 (127.14.241.194:43229), e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051)
I20260928 14:51:07.728567 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.722284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:35651 (local address 127.14.241.194:43229)
0928 14:51:07.722475 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:07.722483 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:07.723451 (+ 968us) server_negotiation.cc:407] Connection header received
0928 14:51:07.723588 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:07.723592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:07.723650 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:07.723719 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:07.724340 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.725061 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.726832 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.727648 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.727815 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.727889 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:07.728152 (+ 263us) server_negotiation.cc:299] Negotiation successful
0928 14:51:07.728275 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 14:51:07.728752 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.722123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:43229 (local address 127.14.241.193:35651)
0928 14:51:07.723334 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:07.723352 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:07.723479 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:07.723757 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:07.723762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:07.723786 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:07.724142 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.724155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.725201 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.725207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.726683 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.726697 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.727836 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.727843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.728007 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.728035 (+ 28us) client_negotiation.cc:769] Sending connection context
0928 14:51:07.728117 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:51:07.728252 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1088,"mutex_wait_us":32,"spinlock_wait_cycles":3328,"thread_start_us":466,"threads_started":1}
I20260928 14:51:07.728852 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.722123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:44051 (local address 127.14.241.193:41347)
0928 14:51:07.723036 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:07.723057 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:07.723224 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:07.723874 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:07.723878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:07.723922 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:07.724267 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.724301 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.725438 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.725441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.726930 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.726942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.728046 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.728052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.728191 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.728212 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 14:51:07.728305 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 14:51:07.728436 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"spinlock_wait_cycles":1920,"thread_start_us":883,"threads_started":1}
I20260928 14:51:07.729223 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.722266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:41347 (local address 127.14.241.195:44051)
0928 14:51:07.723572 (+ 1306us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:07.723578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:07.723595 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:51:07.723660 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:07.723663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:07.723740 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:07.723836 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:07.724478 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.725309 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.727072 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.727832 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.728005 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.728077 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:07.728342 (+ 265us) server_negotiation.cc:299] Negotiation successful
0928 14:51:07.728469 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1183,"thread_start_us":1051,"threads_started":1}
I20260928 14:51:07.729609 23224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4aca40cc49f4250a1ad122412fd8eb7" candidate_uuid: "a93577de632b4d6d919d2c5d9461e6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83c56e4461d4027addc640ce7929367" is_pre_election: true
I20260928 14:51:07.729741 23363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4aca40cc49f4250a1ad122412fd8eb7" candidate_uuid: "a93577de632b4d6d919d2c5d9461e6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" is_pre_election: true
I20260928 14:51:07.729997 23363 raft_consensus.cc:2454] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a93577de632b4d6d919d2c5d9461e6a2 in term 0.
I20260928 14:51:07.730043 23224 raft_consensus.cc:2454] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a93577de632b4d6d919d2c5d9461e6a2 in term 0.
I20260928 14:51:07.730547 23018 leader_election.cc:302] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a93577de632b4d6d919d2c5d9461e6a2, d83c56e4461d4027addc640ce7929367, e2e457bb98dd478caa3cccb3bfd88a4b; no voters:
I20260928 14:51:07.730924 23444 raft_consensus.cc:2781] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:51:07.731032 23444 raft_consensus.cc:482] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:51:07.731117 23444 raft_consensus.cc:3037] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:07.733299 23444 raft_consensus.cc:504] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.733914 23444 leader_election.cc:288] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [CANDIDATE]: Term 1 election: Requested vote from peers d83c56e4461d4027addc640ce7929367 (127.14.241.194:43229), e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051)
I20260928 14:51:07.734333 23363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4aca40cc49f4250a1ad122412fd8eb7" candidate_uuid: "a93577de632b4d6d919d2c5d9461e6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b"
I20260928 14:51:07.734350 23224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4aca40cc49f4250a1ad122412fd8eb7" candidate_uuid: "a93577de632b4d6d919d2c5d9461e6a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83c56e4461d4027addc640ce7929367"
I20260928 14:51:07.734562 23363 raft_consensus.cc:3037] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:07.734544 23224 raft_consensus.cc:3037] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:07.737471 23224 raft_consensus.cc:2454] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a93577de632b4d6d919d2c5d9461e6a2 in term 1.
I20260928 14:51:07.737455 23363 raft_consensus.cc:2454] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a93577de632b4d6d919d2c5d9461e6a2 in term 1.
I20260928 14:51:07.737888 23017 leader_election.cc:302] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a93577de632b4d6d919d2c5d9461e6a2, d83c56e4461d4027addc640ce7929367; no voters:
I20260928 14:51:07.738147 23444 raft_consensus.cc:2781] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:07.738574 23444 raft_consensus.cc:686] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 LEADER]: Becoming Leader. State: Replica: a93577de632b4d6d919d2c5d9461e6a2, State: Running, Role: LEADER
I20260928 14:51:07.738931 23444 consensus_queue.cc:237] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } }
I20260928 14:51:07.742094 22927 catalog_manager.cc:5495] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193). New cstate: current_term: 1 leader_uuid: "a93577de632b4d6d919d2c5d9461e6a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93577de632b4d6d919d2c5d9461e6a2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 44941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 } health_report { overall_health: UNKNOWN } } }
I20260928 14:51:07.797935 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.792447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.14.241.193:44941)
0928 14:51:07.792992 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:07.792996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:07.793016 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:51:07.793098 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:07.793102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:07.793175 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:07.793338 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:51:07.793730 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.794508 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.795803 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.796197 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.796336 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.796793 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:51:07.797384 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:51:07.797503 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:07.797598 (+ 95us) server_negotiation.cc:299] Negotiation successful
0928 14:51:07.797666 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":309,"threads_started":1}
I20260928 14:51:07.813056 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.807853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:50977 (local address 127.14.241.194:43229)
0928 14:51:07.808011 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:07.808018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:07.808565 (+ 547us) server_negotiation.cc:407] Connection header received
0928 14:51:07.808760 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:07.808764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:07.808831 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:07.808905 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:51:07.809405 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.810089 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.811408 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.811800 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.811904 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.812351 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:51:07.812467 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:51:07.812539 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:07.812668 (+ 129us) server_negotiation.cc:299] Negotiation successful
0928 14:51:07.812744 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:51:07.813129 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.807826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:43229 (local address 127.14.241.193:50977)
0928 14:51:07.808458 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:07.808474 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:07.808622 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:07.808933 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:07.808937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:07.808948 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:51:07.809233 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.809243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.810216 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.810221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.811262 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.811276 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.811972 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.811979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.812110 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.812163 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:51:07.812557 (+ 394us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:51:07.812570 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 14:51:07.812639 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 14:51:07.812775 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":287,"threads_started":1}
I20260928 14:51:07.821689 23363 raft_consensus.cc:1261] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Refusing update from remote peer a93577de632b4d6d919d2c5d9461e6a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:07.821941 23224 raft_consensus.cc:1261] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Refusing update from remote peer a93577de632b4d6d919d2c5d9461e6a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:07.822285 23525 consensus_queue.cc:1035] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e457bb98dd478caa3cccb3bfd88a4b" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:07.822634 23444 consensus_queue.cc:1035] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83c56e4461d4027addc640ce7929367" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:07.847950 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.842945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:33095 (local address 127.14.241.193:44941)
0928 14:51:07.843385 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:07.843391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:07.843483 (+ 92us) server_negotiation.cc:407] Connection header received
0928 14:51:07.843649 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:07.843655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:07.843724 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:07.843792 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:51:07.844231 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.845102 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.846342 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.846890 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.847000 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.847342 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:51:07.847443 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:51:07.847521 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:07.847669 (+ 148us) server_negotiation.cc:299] Negotiation successful
0928 14:51:07.847732 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338}
I20260928 14:51:07.848115 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.842840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:44941 (local address 127.14.241.194:33095)
0928 14:51:07.843373 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:07.843394 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:07.843534 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:07.843819 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:07.843823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:07.843845 (+ 22us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:51:07.844076 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.844087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.845243 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.845246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.846218 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:07.846229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.847030 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.847034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:07.847166 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.847200 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:51:07.847539 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:51:07.847551 (+ 12us) client_negotiation.cc:769] Sending connection context
0928 14:51:07.847629 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:51:07.847764 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":244,"threads_started":1}
I20260928 14:51:07.875103 22927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:07.875366 22927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:07.878470 22927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7326e50052ad478fabe0270bb875996b: Sending DeleteTablet for 3 replicas of tablet a4aca40cc49f4250a1ad122412fd8eb7
I20260928 14:51:07.879361 23065 tablet_service.cc:1514] Processing DeleteTablet for tablet a4aca40cc49f4250a1ad122412fd8eb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:07 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:36774
I20260928 14:51:07.879709 23203 tablet_service.cc:1514] Processing DeleteTablet for tablet a4aca40cc49f4250a1ad122412fd8eb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:07 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59146
I20260928 14:51:07.879849 23497 tablet_replica.cc:330] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2: stopping tablet replica
I20260928 14:51:07.880004 23498 tablet_replica.cc:330] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367: stopping tablet replica
I20260928 14:51:07.880035 23497 raft_consensus.cc:2229] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:51:07.880178 23498 raft_consensus.cc:2229] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:07.880343 23343 tablet_service.cc:1514] Processing DeleteTablet for tablet a4aca40cc49f4250a1ad122412fd8eb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:07 UTC) from {username='oryx', principal='oryx/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:60246
I20260928 14:51:07.880409 23498 raft_consensus.cc:2258] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:07.880611 23499 tablet_replica.cc:330] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b: stopping tablet replica
I20260928 14:51:07.880776 23499 raft_consensus.cc:2229] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:07.880970 23499 raft_consensus.cc:2258] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:07.881651 23498 ts_tablet_manager.cc:1905] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:07.881968 23499 ts_tablet_manager.cc:1905] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:07.884738 23497 raft_consensus.cc:2258] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:07.886000 23497 ts_tablet_manager.cc:1905] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:07.888078 23498 ts_tablet_manager.cc:1918] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:51:07.888203 23498 log.cc:1196] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/a4aca40cc49f4250a1ad122412fd8eb7
I20260928 14:51:07.888183 23499 ts_tablet_manager.cc:1918] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:51:07.888332 23499 log.cc:1196] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/a4aca40cc49f4250a1ad122412fd8eb7
I20260928 14:51:07.888631 23498 ts_tablet_manager.cc:1939] T a4aca40cc49f4250a1ad122412fd8eb7 P d83c56e4461d4027addc640ce7929367: Deleting consensus metadata
I20260928 14:51:07.888657 23499 ts_tablet_manager.cc:1939] T a4aca40cc49f4250a1ad122412fd8eb7 P e2e457bb98dd478caa3cccb3bfd88a4b: Deleting consensus metadata
I20260928 14:51:07.889624 22915 catalog_manager.cc:4841] TS e2e457bb98dd478caa3cccb3bfd88a4b (127.14.241.195:44051): tablet a4aca40cc49f4250a1ad122412fd8eb7 (table test-table [id=3c8fa67d09044554a9bf8197ffa23227]) successfully deleted
I20260928 14:51:07.890121 22914 catalog_manager.cc:4841] TS d83c56e4461d4027addc640ce7929367 (127.14.241.194:43229): tablet a4aca40cc49f4250a1ad122412fd8eb7 (table test-table [id=3c8fa67d09044554a9bf8197ffa23227]) successfully deleted
I20260928 14:51:07.890319 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 22999
I20260928 14:51:07.894213 23497 ts_tablet_manager.cc:1918] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:51:07.894361 23497 log.cc:1196] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/a4aca40cc49f4250a1ad122412fd8eb7
I20260928 14:51:07.894801 23497 ts_tablet_manager.cc:1939] T a4aca40cc49f4250a1ad122412fd8eb7 P a93577de632b4d6d919d2c5d9461e6a2: Deleting consensus metadata
I20260928 14:51:07.895973 22914 catalog_manager.cc:4841] TS a93577de632b4d6d919d2c5d9461e6a2 (127.14.241.193:44941): tablet a4aca40cc49f4250a1ad122412fd8eb7 (table test-table [id=3c8fa67d09044554a9bf8197ffa23227]) successfully deleted
Sep 28 14:51:07 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.241.193@KRBTEST.COM
I20260928 14:51:07.906759 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:07.891699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36798 (local address 127.14.241.193:44941)
0928 14:51:07.891887 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:07.891893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:07.892128 (+ 235us) server_negotiation.cc:407] Connection header received
0928 14:51:07.892330 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:07.892335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:07.892397 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:07.892497 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:07.893576 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.894320 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.895538 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:07.896047 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:07.896159 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:07.900936 (+ 4777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:07.900950 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:07.900965 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:07.900997 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:07.903948 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:07.904822 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:07.904830 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:07.904832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:07.904973 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:07.905321 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:07.905324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:07.905326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:07.905718 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:07.905967 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:07.906273 (+ 306us) server_negotiation.cc:299] Negotiation successful
0928 14:51:07.906438 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:51:07.907444 23130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:08.013435 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 22999
I20260928 14:51:08.029230 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 23139
Sep 28 14:51:08 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.241.194@KRBTEST.COM
I20260928 14:51:08.043560 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:08.030056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.14.241.194:43229)
0928 14:51:08.030239 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:08.030244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:08.030261 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:51:08.030316 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:08.030321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:08.030386 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:08.030466 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:08.031590 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:08.032277 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:08.033535 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:08.033969 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:08.034105 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:08.038085 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:08.038099 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:08.038114 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:08.038148 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:08.040976 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:08.041622 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:08.041630 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:08.041632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:08.041802 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:08.042070 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:08.042073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:08.042075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:08.042410 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:08.042648 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:08.043068 (+ 420us) server_negotiation.cc:299] Negotiation successful
0928 14:51:08.043241 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 14:51:08.044188 23269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:08.188815 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 23139
I20260928 14:51:08.210808 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 23278
Sep 28 14:51:08 dist-test-slave-vk40 krb5kdc[22882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.241.195@KRBTEST.COM
I20260928 14:51:08.225394 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:08.211568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60262 (local address 127.14.241.195:44051)
0928 14:51:08.211722 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:08.211726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:08.211763 (+ 37us) server_negotiation.cc:407] Connection header received
0928 14:51:08.211921 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:08.211925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:08.211997 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:08.212080 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:08.213353 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:08.214063 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:08.215206 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:08.215591 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:08.215677 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:08.219833 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:08.219849 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:08.219864 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:08.219905 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:08.222871 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:08.223586 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:08.223593 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:08.223595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:08.223746 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:08.224045 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:08.224048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:08.224050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:08.224441 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:08.224655 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:08.224952 (+ 297us) server_negotiation.cc:299] Negotiation successful
0928 14:51:08.225124 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 14:51:08.226097 23408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:08.319928 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 23278
I20260928 14:51:08.334347 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 22896
I20260928 14:51:08.335561 22968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:08.453006 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 22896
2026-09-28T14:51:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5518 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:51:08 dist-test-slave-vk40 krb5kdc[23544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:51:08 dist-test-slave-vk40 krb5kdc[23544](info): set up 2 sockets
Sep 28 14:51:08 dist-test-slave-vk40 krb5kdc[23544](info): commencing operation
krb5kdc: starting...
W20260928 14:51:09.467980 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:51:09 dist-test-slave-vk40 krb5kdc[23544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:51:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:51:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:51:09.677675 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:40909
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:46257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:40909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:09.997277 23557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:09.997563 23557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:09.997661 23557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:10.004820 23557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:51:10.004925 23557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:10.004969 23557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:10.005023 23557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:51:10.005069 23557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:51:10.015384 23557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:46257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:40909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:40909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:10.016642 23557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:10.017930 23557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:10.029105 23565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:10.029168 23563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:10.029130 23562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:10.030308 23557 server_base.cc:1034] running on GCE node
I20260928 14:51:10.031015 23557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:10.032423 23557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:10.033625 23557 hybrid_clock.cc:648] HybridClock initialized: now 1790607070033565 us; error 50 us; skew 500 ppm
I20260928 14:51:10.033936 23557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-28T14:51:10Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1644 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:51:10 dist-test-slave-vk40 krb5kdc[23571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:51:10 dist-test-slave-vk40 krb5kdc[23571](info): set up 2 sockets
Sep 28 14:51:10 dist-test-slave-vk40 krb5kdc[23571](info): commencing operation
krb5kdc: starting...
W20260928 14:51:11.131732 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.007s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:51:11 dist-test-slave-vk40 krb5kdc[23571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:51:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:51:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:51:11.348201 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38777
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:37647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:38777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:11.667588 23584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:11.667878 23584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:11.668004 23584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:11.675521 23584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:51:11.675642 23584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:11.675719 23584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:11.675796 23584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:51:11.675853 23584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:51:11.686009 23584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:37647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:38777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:11.687345 23584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:11.688638 23584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:11.700189 23590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:11.700309 23592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:11.700727 23589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:11.701136 23584 server_base.cc:1034] running on GCE node
I20260928 14:51:11.701943 23584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:11.703368 23584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:11.704586 23584 hybrid_clock.cc:648] HybridClock initialized: now 1790607071704532 us; error 35 us; skew 500 ppm
I20260928 14:51:11.704958 23584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:11 dist-test-slave-vk40 krb5kdc[23571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607071, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:11.714042 23584 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:51:11.715595 23584 webserver.cc:460] Webserver started at http://127.14.241.254:37581/ using document root <none> and password file <none>
I20260928 14:51:11.716334 23584 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:11.716460 23584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:11.716776 23584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:11.719846 23584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "fd989e0c1edd45398b10826d42b7d02f"
format_stamp: "Formatted at 2026-09-28 14:51:11 on dist-test-slave-vk40"
server_key: "be0fc90825e8456d056ea3f11e5aef9a"
server_key_iv: "73ea6269b668f0fb51509e5bfdaf23ec"
server_key_version: "encryptionkey@0"
I20260928 14:51:11.720635 23584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "fd989e0c1edd45398b10826d42b7d02f"
format_stamp: "Formatted at 2026-09-28 14:51:11 on dist-test-slave-vk40"
server_key: "be0fc90825e8456d056ea3f11e5aef9a"
server_key_iv: "73ea6269b668f0fb51509e5bfdaf23ec"
server_key_version: "encryptionkey@0"
I20260928 14:51:11.725827 23584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260928 14:51:11.729467 23599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:11.730885 23584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 14:51:11.731086 23584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "fd989e0c1edd45398b10826d42b7d02f"
format_stamp: "Formatted at 2026-09-28 14:51:11 on dist-test-slave-vk40"
server_key: "be0fc90825e8456d056ea3f11e5aef9a"
server_key_iv: "73ea6269b668f0fb51509e5bfdaf23ec"
server_key_version: "encryptionkey@0"
I20260928 14:51:11.731289 23584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:11.747977 23584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:11.752322 23584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:11.752606 23584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:11.770089 23584 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:38777
I20260928 14:51:11.770097 23650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:38777 every 8 connection(s)
I20260928 14:51:11.771472 23584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:51:11.775220 23651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:11.779040 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 23584
I20260928 14:51:11.779261 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:51:11.780051 15303 external_mini_cluster.cc:1428] Setting key 9425e3220fc26f472f4489db3470c5b0
I20260928 14:51:11.789453 23651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f: Bootstrap starting.
I20260928 14:51:11.793404 23651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:11.794847 23651 log.cc:824] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f: Log is configured to *not* fsync() on all Append() calls
Sep 28 14:51:11 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:51:11.798303 23651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f: No bootstrap required, opened a new log
I20260928 14:51:11.805011 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:11.782380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.14.241.254:38777)
0928 14:51:11.783211 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:11.783224 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:11.783266 (+ 42us) server_negotiation.cc:407] Connection header received
0928 14:51:11.785087 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:11.785115 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:11.785611 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:11.786016 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:11.787907 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:11.789364 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:11.791119 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:11.791891 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:11.792034 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:11.797269 (+ 5235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:11.797279 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:11.797295 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:11.797351 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:11.800483 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:11.801408 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:11.801416 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:11.801422 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:11.801611 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:11.802062 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:11.802067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:11.802072 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:11.802683 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:11.802947 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:11.803397 (+ 450us) server_negotiation.cc:299] Negotiation successful
0928 14:51:11.803826 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":396,"threads_started":1}
I20260928 14:51:11.807309 23651 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } }
I20260928 14:51:11.807682 23651 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:11.807780 23651 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd989e0c1edd45398b10826d42b7d02f, State: Initialized, Role: FOLLOWER
I20260928 14:51:11.808439 23651 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } }
I20260928 14:51:11.808665 23651 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:51:11.808785 23651 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:51:11.808929 23651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:11.811034 23651 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } }
I20260928 14:51:11.811524 23651 leader_election.cc:302] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd989e0c1edd45398b10826d42b7d02f; no voters:
I20260928 14:51:11.812077 23651 leader_election.cc:288] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:51:11.812206 23656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:11.813378 23656 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 1 LEADER]: Becoming Leader. State: Replica: fd989e0c1edd45398b10826d42b7d02f, State: Running, Role: LEADER
I20260928 14:51:11.813973 23656 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } }
I20260928 14:51:11.814280 23651 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:51:11.816938 23657 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd989e0c1edd45398b10826d42b7d02f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } } }
I20260928 14:51:11.817335 23657 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:11.818101 23658 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd989e0c1edd45398b10826d42b7d02f. Latest consensus state: current_term: 1 leader_uuid: "fd989e0c1edd45398b10826d42b7d02f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } } }
I20260928 14:51:11.818441 23658 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:11.821375 23665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:51:11.825838 23665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:51:11.834534 23665 catalog_manager.cc:1347] Generated new cluster ID: 0b75ee6a9df24d4dbd1052d5a32fa8e0
I20260928 14:51:11.834652 23665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:51:11.841492 23665 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:51:11.843428 23665 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:51:11.858946 23665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f: Generated new TSK 0
I20260928 14:51:11.859862 23665 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:51:11.912552 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38777
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:37647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:12.237088 23677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:12.237380 23677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:12.237514 23677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:12.244916 23677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:12.245033 23677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:12.245185 23677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:51:12.255108 23677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:37647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:12.256609 23677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:12.257927 23677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:12.271121 23685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:12.271149 23683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:12.271291 23682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:12.271950 23677 server_base.cc:1034] running on GCE node
I20260928 14:51:12.272620 23677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:12.273564 23677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:12.274763 23677 hybrid_clock.cc:648] HybridClock initialized: now 1790607072274706 us; error 55 us; skew 500 ppm
I20260928 14:51:12.275067 23677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:12 dist-test-slave-vk40 krb5kdc[23571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607072, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:12.278129 23677 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:51:12.279511 23677 webserver.cc:460] Webserver started at http://127.14.241.193:34009/ using document root <none> and password file <none>
I20260928 14:51:12.280179 23677 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:12.280320 23677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:12.280612 23677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:12.283469 23677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "4da5bcdcca624b678249d89ffa7c2836"
format_stamp: "Formatted at 2026-09-28 14:51:12 on dist-test-slave-vk40"
server_key: "861c1c25e616e93b6cbc00b1809ea088"
server_key_iv: "404e4df14a877a86236d850fcc214cd3"
server_key_version: "encryptionkey@0"
I20260928 14:51:12.284214 23677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "4da5bcdcca624b678249d89ffa7c2836"
format_stamp: "Formatted at 2026-09-28 14:51:12 on dist-test-slave-vk40"
server_key: "861c1c25e616e93b6cbc00b1809ea088"
server_key_iv: "404e4df14a877a86236d850fcc214cd3"
server_key_version: "encryptionkey@0"
I20260928 14:51:12.289297 23677 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260928 14:51:12.292656 23692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:12.294121 23677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 14:51:12.294296 23677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "4da5bcdcca624b678249d89ffa7c2836"
format_stamp: "Formatted at 2026-09-28 14:51:12 on dist-test-slave-vk40"
server_key: "861c1c25e616e93b6cbc00b1809ea088"
server_key_iv: "404e4df14a877a86236d850fcc214cd3"
server_key_version: "encryptionkey@0"
I20260928 14:51:12.294482 23677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:12.346282 23677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:12.350420 23677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:12.350669 23677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:12.351574 23677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:12.352926 23677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:12.353005 23677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:12.353118 23677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:12.353168 23677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:12.383374 23677 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:43275
I20260928 14:51:12.383438 23804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:43275 every 8 connection(s)
I20260928 14:51:12.384651 23677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:51:12.394062 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 23677
I20260928 14:51:12.394233 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:51:12.394846 15303 external_mini_cluster.cc:1428] Setting key ac36360fcc3cc31146962a9baab48aa2
Sep 28 14:51:12 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607072, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:51:12.405848 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:12.387143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:59865 (local address 127.14.241.254:38777)
0928 14:51:12.387638 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:12.387644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:12.388566 (+ 922us) server_negotiation.cc:407] Connection header received
0928 14:51:12.390338 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:12.390342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:12.390412 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:12.390507 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:12.391955 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:12.392725 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:12.394288 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:12.394724 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:12.394844 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:12.399472 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:12.399478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:12.399482 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:12.399525 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:12.402700 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:12.403519 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:12.403525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:12.403528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:12.403676 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:12.404084 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:12.404089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:12.404092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:12.404449 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:12.404591 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:12.405392 (+ 801us) server_negotiation.cc:299] Negotiation successful
0928 14:51:12.405580 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":231,"threads_started":1}
I20260928 14:51:12.407310 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:12.387694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:59865)
0928 14:51:12.388422 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:12.388461 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:12.390069 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:12.390710 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:12.390720 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:12.391245 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:12.391813 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:12.391831 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:12.392886 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:12.392890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:12.394133 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:12.394147 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:12.394961 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:12.394968 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:12.395221 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:12.395979 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:12.396016 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:12.399155 (+ 3139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:12.402877 (+ 3722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:12.402886 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:12.402901 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:12.403394 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:12.403778 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:12.403781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:12.403784 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:12.403990 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:12.404584 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:12.404591 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:12.404964 (+ 373us) client_negotiation.cc:769] Sending connection context
0928 14:51:12.405308 (+ 344us) client_negotiation.cc:240] Negotiation successful
0928 14:51:12.405905 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":254,"threads_started":1}
I20260928 14:51:12.409114 23805 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38777
I20260928 14:51:12.409475 23805 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:12.410318 23805 heartbeater.cc:511] Master 127.14.241.254:38777 requested a full tablet report, sending...
I20260928 14:51:12.412817 23616 ts_manager.cc:194] Registered new tserver with Master: 4da5bcdcca624b678249d89ffa7c2836 (127.14.241.193:43275)
I20260928 14:51:12.414837 23616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:59865
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:51:12.447722 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38777
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:37647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:12.765147 23811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:12.765435 23811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:12.765558 23811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:12.772907 23811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:12.773037 23811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:12.773190 23811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:51:12.783268 23811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:37647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:12.784579 23811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:12.785856 23811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:12.798386 23816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:12.798377 23817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:12.798488 23819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:12.799273 23811 server_base.cc:1034] running on GCE node
I20260928 14:51:12.799860 23811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:12.800823 23811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:12.802031 23811 hybrid_clock.cc:648] HybridClock initialized: now 1790607072801976 us; error 40 us; skew 500 ppm
I20260928 14:51:12.802342 23811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:12 dist-test-slave-vk40 krb5kdc[23571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607072, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:12.805336 23811 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:51:12.806770 23811 webserver.cc:460] Webserver started at http://127.14.241.194:32831/ using document root <none> and password file <none>
I20260928 14:51:12.807426 23811 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:12.807519 23811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:12.807767 23811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:12.810637 23811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "5aa4d7e355784b0b84b2a0574acd7ebd"
format_stamp: "Formatted at 2026-09-28 14:51:12 on dist-test-slave-vk40"
server_key: "6643738098f40b0ce93a05e01974f210"
server_key_iv: "97dc72b2510f01e18fefade2befa404e"
server_key_version: "encryptionkey@0"
I20260928 14:51:12.811357 23811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "5aa4d7e355784b0b84b2a0574acd7ebd"
format_stamp: "Formatted at 2026-09-28 14:51:12 on dist-test-slave-vk40"
server_key: "6643738098f40b0ce93a05e01974f210"
server_key_iv: "97dc72b2510f01e18fefade2befa404e"
server_key_version: "encryptionkey@0"
I20260928 14:51:12.816444 23811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260928 14:51:12.819957 23826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:12.821399 23811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:51:12.821540 23811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "5aa4d7e355784b0b84b2a0574acd7ebd"
format_stamp: "Formatted at 2026-09-28 14:51:12 on dist-test-slave-vk40"
server_key: "6643738098f40b0ce93a05e01974f210"
server_key_iv: "97dc72b2510f01e18fefade2befa404e"
server_key_version: "encryptionkey@0"
I20260928 14:51:12.821727 23811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:12.837155 23811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:12.841156 23811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:12.841418 23811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:12.842352 23811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:12.843619 23811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:12.843688 23811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:12.843779 23811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:12.843837 23811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:12.873080 23811 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:39185
I20260928 14:51:12.873117 23938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:39185 every 8 connection(s)
I20260928 14:51:12.874390 23811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:51:12.878103 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 23811
I20260928 14:51:12.878281 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:51:12.878821 15303 external_mini_cluster.cc:1428] Setting key 4c6959aab2de2126c3102fca335ed83a
Sep 28 14:51:12 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607072, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:51:12.895958 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:12.876743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:36661 (local address 127.14.241.254:38777)
0928 14:51:12.876963 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:12.876969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:12.878235 (+ 1266us) server_negotiation.cc:407] Connection header received
0928 14:51:12.880484 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:12.880489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:12.880568 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:12.880672 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:12.882592 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:12.883277 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:12.885308 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:12.885731 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:12.885825 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:12.889806 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:12.889813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:12.889818 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:12.889858 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:12.892933 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:12.893678 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:12.893683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:12.893686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:12.893835 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:12.894231 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:12.894236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:12.894240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:12.894574 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:12.894716 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:12.895431 (+ 715us) server_negotiation.cc:299] Negotiation successful
0928 14:51:12.895655 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 14:51:12.897064 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:12.877295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:36661)
0928 14:51:12.878053 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:12.878116 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:12.880183 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:12.880922 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:12.880934 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:12.881645 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:12.882411 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:12.882434 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:12.883433 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:12.883438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:12.885179 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:12.885193 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:12.885931 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:12.885938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:12.886230 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:12.886945 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:12.886971 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:12.889551 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:12.893088 (+ 3537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:12.893096 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:12.893110 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:12.893554 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:12.893931 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:12.893934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:12.893936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:12.894125 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:12.894671 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:12.894676 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:12.895011 (+ 335us) client_negotiation.cc:769] Sending connection context
0928 14:51:12.895305 (+ 294us) client_negotiation.cc:240] Negotiation successful
0928 14:51:12.895801 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":257,"threads_started":1}
I20260928 14:51:12.898772 23939 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38777
I20260928 14:51:12.899070 23939 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:12.899830 23939 heartbeater.cc:511] Master 127.14.241.254:38777 requested a full tablet report, sending...
I20260928 14:51:12.901484 23616 ts_manager.cc:194] Registered new tserver with Master: 5aa4d7e355784b0b84b2a0574acd7ebd (127.14.241.194:39185)
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
I20260928 14:51:12.902595 23616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:36661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:51:12.929435 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38777
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:37647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:13.252213 23944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:13.252523 23944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:13.252645 23944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:13.259892 23944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:13.260005 23944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:13.260149 23944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:51:13.269833 23944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:37647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:13.271097 23944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:13.272543 23944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:13.285741 23950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:13.285691 23952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:13.285779 23949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:13.286540 23944 server_base.cc:1034] running on GCE node
I20260928 14:51:13.287148 23944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:13.288070 23944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:13.289292 23944 hybrid_clock.cc:648] HybridClock initialized: now 1790607073289218 us; error 55 us; skew 500 ppm
I20260928 14:51:13.289626 23944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:13 dist-test-slave-vk40 krb5kdc[23571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607073, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:13.292762 23944 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:51:13.294269 23944 webserver.cc:460] Webserver started at http://127.14.241.195:39635/ using document root <none> and password file <none>
I20260928 14:51:13.294927 23944 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:13.295023 23944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:13.295275 23944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:13.298182 23944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "f8536995377c40b4bc49c13961b0d08b"
format_stamp: "Formatted at 2026-09-28 14:51:13 on dist-test-slave-vk40"
server_key: "d8fccab7dce3789606aede299982137a"
server_key_iv: "c47859c4347d756d41208dcec50a5bca"
server_key_version: "encryptionkey@0"
I20260928 14:51:13.299026 23944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "f8536995377c40b4bc49c13961b0d08b"
format_stamp: "Formatted at 2026-09-28 14:51:13 on dist-test-slave-vk40"
server_key: "d8fccab7dce3789606aede299982137a"
server_key_iv: "c47859c4347d756d41208dcec50a5bca"
server_key_version: "encryptionkey@0"
I20260928 14:51:13.304251 23944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 14:51:13.307668 23959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:13.309054 23944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:51:13.309198 23944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "f8536995377c40b4bc49c13961b0d08b"
format_stamp: "Formatted at 2026-09-28 14:51:13 on dist-test-slave-vk40"
server_key: "d8fccab7dce3789606aede299982137a"
server_key_iv: "c47859c4347d756d41208dcec50a5bca"
server_key_version: "encryptionkey@0"
I20260928 14:51:13.309382 23944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:13.336606 23944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:13.340643 23944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:13.340909 23944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:13.341864 23944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:13.343206 23944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:13.343287 23944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:13.343410 23944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:13.343458 23944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:13.373217 23944 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:32781
I20260928 14:51:13.373288 24071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:32781 every 8 connection(s)
I20260928 14:51:13.374476 23944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:51:13.380323 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 23944
I20260928 14:51:13.380492 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:51:13.380905 15303 external_mini_cluster.cc:1428] Setting key f2d6e09df6c952bc2c84f403b3a83950
Sep 28 14:51:13 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607073, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:51:13.394845 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:13.376954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:45345 (local address 127.14.241.254:38777)
0928 14:51:13.377133 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:13.377137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:13.378325 (+ 1188us) server_negotiation.cc:407] Connection header received
0928 14:51:13.380159 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:13.380163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:13.380249 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:13.380383 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:13.382073 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:13.382987 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:13.385254 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:13.385669 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:13.385769 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:13.389613 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:13.389618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:13.389621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:13.389655 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:13.392086 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:13.392751 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:13.392755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:13.392757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:13.392881 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:13.393216 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:13.393219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:13.393221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:13.393533 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:13.393682 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:13.394308 (+ 626us) server_negotiation.cc:299] Negotiation successful
0928 14:51:13.394511 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 14:51:13.395777 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:13.377490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:45345)
0928 14:51:13.378149 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:13.378193 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:13.379821 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:13.380636 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:13.380645 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:13.381192 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:13.381886 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:13.381908 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:13.383140 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:13.383145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:13.385108 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:13.385130 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:13.385888 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:13.385896 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:13.386171 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:13.386876 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:13.386904 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:13.389400 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:13.392196 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:13.392202 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:13.392214 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:13.392662 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:13.392961 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:13.392964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:13.392967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:13.393137 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:13.393639 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:13.393644 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:13.393951 (+ 307us) client_negotiation.cc:769] Sending connection context
0928 14:51:13.394168 (+ 217us) client_negotiation.cc:240] Negotiation successful
0928 14:51:13.394637 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":251,"threads_started":1}
I20260928 14:51:13.397048 24072 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38777
I20260928 14:51:13.397357 24072 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:13.398113 24072 heartbeater.cc:511] Master 127.14.241.254:38777 requested a full tablet report, sending...
I20260928 14:51:13.399564 23616 ts_manager.cc:194] Registered new tserver with Master: f8536995377c40b4bc49c13961b0d08b (127.14.241.195:32781)
I20260928 14:51:13.400388 23616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:45345
I20260928 14:51:13.408499 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:51:13.418388 23805 heartbeater.cc:503] Master 127.14.241.254:38777 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab.
Entry for principal oryx/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab.
WARNING: no policy specified for oryx/127.14.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab.
Entry for principal oryx/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab.
WARNING: no policy specified for oryx/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab.
Entry for principal oryx/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab.
I20260928 14:51:13.543314 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 23677
Sep 28 14:51:13 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:51:13.558928 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:13.544505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42488 (local address 127.14.241.193:43275)
0928 14:51:13.545130 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:13.545137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:13.545162 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:51:13.545223 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:13.545229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:13.545453 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:13.545583 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:13.546977 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:13.547962 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:13.549290 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:13.549854 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:13.549949 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:13.553766 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:13.553778 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:13.553793 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:13.553834 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:13.556519 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:13.557173 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:13.557180 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:13.557183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:13.557306 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:13.557605 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:13.557608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:13.557610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:13.557932 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:13.558165 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:13.558389 (+ 224us) server_negotiation.cc:299] Negotiation successful
0928 14:51:13.558582 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":306,"threads_started":1}
I20260928 14:51:13.560189 23800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:13.642409 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 23677
I20260928 14:51:13.656327 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 23811
Sep 28 14:51:13 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:51:13.671501 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:13.657205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35864 (local address 127.14.241.194:39185)
0928 14:51:13.657664 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:13.657671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:13.657696 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:51:13.657896 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:13.657907 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:13.658141 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:13.658282 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:13.659468 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:13.660563 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:13.661795 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:13.662368 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:13.662484 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:13.666403 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:13.666416 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:13.666431 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:13.666466 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:13.669228 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:13.669849 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:13.669856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:13.669859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:13.669982 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:13.670277 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:13.670280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:13.670282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:13.670610 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:13.670867 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:13.671018 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 14:51:13.671218 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":223,"threads_started":1}
I20260928 14:51:13.672931 23934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:13.753496 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 23811
I20260928 14:51:13.766583 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 23944
Sep 28 14:51:13 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:51:13.783339 24086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:13.767543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56678 (local address 127.14.241.195:32781)
0928 14:51:13.768145 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:13.768151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:13.768179 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:51:13.768237 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:13.768243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:13.768505 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:13.768633 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:13.769844 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:13.771444 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:13.772795 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:13.773360 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:13.773481 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:13.777918 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:13.777931 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:13.777946 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:13.777983 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:13.780868 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:13.781575 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:13.781583 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:13.781586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:13.781733 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:13.782059 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:13.782063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:13.782065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:13.782430 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:13.782671 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:13.782854 (+ 183us) server_negotiation.cc:299] Negotiation successful
0928 14:51:13.783049 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":309,"threads_started":1}
I20260928 14:51:13.784584 24067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:13.865767 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 23944
I20260928 14:51:13.878708 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 23584
I20260928 14:51:13.879882 23646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:13.982692 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 23584
I20260928 14:51:13.999822 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38777
--webserver_interface=127.14.241.254
--webserver_port=37581
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:37647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:38777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:14.319559 24089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:14.319873 24089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:14.320001 24089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:14.327346 24089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:51:14.327481 24089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:14.327564 24089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:14.327641 24089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:51:14.327700 24089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:51:14.337860 24089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:37647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:38777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=37581
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:14.339170 24089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:14.340593 24089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:14.352275 24094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:14.352227 24095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:14.352478 24097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:14.352965 24089 server_base.cc:1034] running on GCE node
I20260928 14:51:14.353722 24089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:14.355202 24089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:14.356410 24089 hybrid_clock.cc:648] HybridClock initialized: now 1790607074356360 us; error 35 us; skew 500 ppm
I20260928 14:51:14.356745 24089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:14 dist-test-slave-vk40 krb5kdc[23571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607074, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:14.359858 24089 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:51:14.361429 24089 webserver.cc:460] Webserver started at http://127.14.241.254:37581/ using document root <none> and password file <none>
I20260928 14:51:14.362110 24089 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:14.362233 24089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:14.367791 24089 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260928 14:51:14.370678 24104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:14.372020 24089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 14:51:14.372177 24089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "fd989e0c1edd45398b10826d42b7d02f"
format_stamp: "Formatted at 2026-09-28 14:51:11 on dist-test-slave-vk40"
server_key: "be0fc90825e8456d056ea3f11e5aef9a"
server_key_iv: "73ea6269b668f0fb51509e5bfdaf23ec"
server_key_version: "encryptionkey@0"
I20260928 14:51:14.373222 24089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:14.397835 24089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:14.402170 24089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:14.402442 24089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:14.419965 24089 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:38777
I20260928 14:51:14.419994 24155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:38777 every 8 connection(s)
I20260928 14:51:14.421499 24089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:51:14.428687 24156 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:51:14.430176 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 24089
I20260928 14:51:14.430928 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:43275
--local_ip_for_outbound_sockets=127.14.241.193
--tserver_master_addrs=127.14.241.254:38777
--webserver_port=34009
--webserver_interface=127.14.241.193
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:37647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.241.193
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:51:14.432116 24156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f: Bootstrap starting.
I20260928 14:51:14.456174 24156 log.cc:824] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:51:14.467432 24156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:51:14.468047 24156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f: Bootstrap complete.
I20260928 14:51:14.481113 24156 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } }
I20260928 14:51:14.482316 24156 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd989e0c1edd45398b10826d42b7d02f, State: Initialized, Role: FOLLOWER
I20260928 14:51:14.483050 24156 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } }
I20260928 14:51:14.483325 24156 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:51:14.483453 24156 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:51:14.483618 24156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:51:14.489307 24156 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } }
I20260928 14:51:14.490103 24156 leader_election.cc:302] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd989e0c1edd45398b10826d42b7d02f; no voters:
I20260928 14:51:14.490787 24156 leader_election.cc:288] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:51:14.490937 24160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:51:14.492057 24160 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [term 2 LEADER]: Becoming Leader. State: Replica: fd989e0c1edd45398b10826d42b7d02f, State: Running, Role: LEADER
I20260928 14:51:14.492648 24160 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } }
I20260928 14:51:14.492946 24156 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:51:14.494920 24162 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd989e0c1edd45398b10826d42b7d02f. Latest consensus state: current_term: 2 leader_uuid: "fd989e0c1edd45398b10826d42b7d02f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } } }
I20260928 14:51:14.495260 24162 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:14.495698 24161 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fd989e0c1edd45398b10826d42b7d02f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd989e0c1edd45398b10826d42b7d02f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38777 } } }
I20260928 14:51:14.496003 24161 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:14.499111 24165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:51:14.505107 24165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:51:14.506558 24165 catalog_manager.cc:1259] Loaded cluster ID: 0b75ee6a9df24d4dbd1052d5a32fa8e0
I20260928 14:51:14.506743 24165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:51:14.510823 24165 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:51:14.512346 24165 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fd989e0c1edd45398b10826d42b7d02f: Loaded TSK: 0
I20260928 14:51:14.513986 24165 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:51:14.780488 24158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:14.780774 24158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:14.780906 24158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:14.788266 24158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:14.788425 24158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:14.788582 24158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:51:14.798714 24158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:37647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.193.keytab
--principal=oryx/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:43275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=34009
--tserver_master_addrs=127.14.241.254:38777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:14.800066 24158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:14.801432 24158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:14.815874 24182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:14.815799 24185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:14.815824 24183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:14.817690 24158 server_base.cc:1034] running on GCE node
I20260928 14:51:14.818322 24158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:14.819342 24158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:14.820571 24158 hybrid_clock.cc:648] HybridClock initialized: now 1790607074820513 us; error 45 us; skew 500 ppm
I20260928 14:51:14.820890 24158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:14 dist-test-slave-vk40 krb5kdc[23571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607074, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:14.823923 24158 init.cc:377] Logged in from keytab as oryx/127.14.241.193@KRBTEST.COM (short username oryx)
I20260928 14:51:14.825552 24158 webserver.cc:460] Webserver started at http://127.14.241.193:34009/ using document root <none> and password file <none>
I20260928 14:51:14.826274 24158 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:14.826393 24158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:14.831954 24158 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260928 14:51:14.834904 24192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:14.836460 24158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:51:14.836629 24158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "4da5bcdcca624b678249d89ffa7c2836"
format_stamp: "Formatted at 2026-09-28 14:51:12 on dist-test-slave-vk40"
server_key: "861c1c25e616e93b6cbc00b1809ea088"
server_key_iv: "404e4df14a877a86236d850fcc214cd3"
server_key_version: "encryptionkey@0"
I20260928 14:51:14.837589 24158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:14.863528 24158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:14.867923 24158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:14.868176 24158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:14.869189 24158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:14.870589 24158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:14.870679 24158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:14.870779 24158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:14.870850 24158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:14.900962 24158 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:43275
I20260928 14:51:14.901010 24304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:43275 every 8 connection(s)
I20260928 14:51:14.902390 24158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:51:14.906839 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 24158
I20260928 14:51:14.907428 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:39185
--local_ip_for_outbound_sockets=127.14.241.194
--tserver_master_addrs=127.14.241.254:38777
--webserver_port=32831
--webserver_interface=127.14.241.194
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:37647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.241.194
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:51:14 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:14 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:14.920686 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:14.905373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:60403)
0928 14:51:14.906194 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:14.906261 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:14.908074 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:14.909680 (+ 1606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:14.909691 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:14.910298 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:14.910858 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:14.910877 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.912524 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.912528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:14.913833 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:14.913847 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.914913 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.914920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:14.915177 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:14.915819 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:14.915843 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:14.919309 (+ 3466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":322,"threads_started":1}
W20260928 14:51:14.920746 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:14.905266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:60403 (local address 127.14.241.254:38777)
0928 14:51:14.906299 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:14.906312 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:14.906370 (+ 58us) server_negotiation.cc:407] Connection header received
0928 14:51:14.908725 (+ 2355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:14.908741 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:14.909139 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:14.909504 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:14.911052 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.912363 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.913984 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.914665 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.914855 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:14.919715 (+ 4860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:60403: BlockingRecv error: recv got EOF from 127.14.241.193:60403 (error 108)
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":555,"threads_started":1}
W20260928 14:51:14.921579 24305 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:38777 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.241.254:38777: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Sep 28 14:51:14 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:14 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:14.930738 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:14.922185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:51981)
0928 14:51:14.922406 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:14.922425 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:14.922545 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:14.922848 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:14.922852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:14.923256 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:14.923465 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:14.923476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.924607 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.924613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:14.926028 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:14.926044 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.926696 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.926702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:14.926881 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:14.927485 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:14.927508 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:14.930505 (+ 2997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":74}
W20260928 14:51:14.930977 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:14.922432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:51981 (local address 127.14.241.254:38777)
0928 14:51:14.922569 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:14.922573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:14.922608 (+ 35us) server_negotiation.cc:407] Connection header received
0928 14:51:14.922658 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:14.922661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:14.922719 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:14.922802 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:14.923608 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.924470 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.926163 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.926546 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.926633 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:14.930753 (+ 4120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:51981: BlockingRecv error: recv got EOF from 127.14.241.193:51981 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 28 14:51:14 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:14 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:14.939122 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:14.931451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:60711)
0928 14:51:14.931609 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:14.931624 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:14.931708 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:14.931981 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:14.931984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:14.932251 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:14.932445 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:14.932453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.933823 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.933827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:14.934744 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:14.934756 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.935615 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.935619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:14.935744 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:14.936346 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:14.936364 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:14.938904 (+ 2540us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260928 14:51:14.939420 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:14.931516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:60711 (local address 127.14.241.254:38777)
0928 14:51:14.931705 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:14.931712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:14.931730 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:51:14.931784 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:14.931790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:14.931875 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:14.931969 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:14.932560 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.933650 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.934867 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:14.935496 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:14.935613 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:14.939160 (+ 3547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:60711: BlockingRecv error: recv got EOF from 127.14.241.193:60711 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260928 14:51:14.939479 24305 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:51:15.225484 24309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:15.225767 24309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:15.225868 24309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:15.233048 24309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:15.233162 24309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:15.233304 24309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:51:15.243026 24309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:37647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.194.keytab
--principal=oryx/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:39185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=32831
--tserver_master_addrs=127.14.241.254:38777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:15.244391 24309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:15.245675 24309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:15.258548 24314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:15.258642 24317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:15.258842 24315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:15.259532 24309 server_base.cc:1034] running on GCE node
I20260928 14:51:15.260143 24309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:15.261116 24309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:15.262385 24309 hybrid_clock.cc:648] HybridClock initialized: now 1790607075262316 us; error 54 us; skew 500 ppm
I20260928 14:51:15.262719 24309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607075, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:15.265805 24309 init.cc:377] Logged in from keytab as oryx/127.14.241.194@KRBTEST.COM (short username oryx)
I20260928 14:51:15.267401 24309 webserver.cc:460] Webserver started at http://127.14.241.194:32831/ using document root <none> and password file <none>
I20260928 14:51:15.268095 24309 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:15.268216 24309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:15.273522 24309 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260928 14:51:15.276155 24324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:15.277593 24309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:51:15.277738 24309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "5aa4d7e355784b0b84b2a0574acd7ebd"
format_stamp: "Formatted at 2026-09-28 14:51:12 on dist-test-slave-vk40"
server_key: "6643738098f40b0ce93a05e01974f210"
server_key_iv: "97dc72b2510f01e18fefade2befa404e"
server_key_version: "encryptionkey@0"
I20260928 14:51:15.278644 24309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:15.309392 24309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:15.313364 24309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:15.313589 24309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:15.314514 24309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:15.315797 24309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:15.315881 24309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:15.315958 24309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:15.316008 24309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:15.345971 24309 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:39185
I20260928 14:51:15.346025 24436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:39185 every 8 connection(s)
I20260928 14:51:15.347376 24309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:51:15.347666 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 24309
I20260928 14:51:15.348337 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:32781
--local_ip_for_outbound_sockets=127.14.241.195
--tserver_master_addrs=127.14.241.254:38777
--webserver_port=39635
--webserver_interface=127.14.241.195
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:37647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.241.195
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 14:51:15.364173 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:15.350046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:34591 (local address 127.14.241.254:38777)
0928 14:51:15.350245 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:15.350251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:15.351474 (+ 1223us) server_negotiation.cc:407] Connection header received
0928 14:51:15.353860 (+ 2386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:15.353864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:15.353947 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:15.354052 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:15.355896 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.356789 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.358353 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.358769 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.358886 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.363904 (+ 5018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:34591: BlockingRecv error: recv got EOF from 127.14.241.194:34591 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:51:15.364913 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:15.350516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:34591)
0928 14:51:15.351292 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:15.351341 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:15.353531 (+ 2190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:15.354308 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:15.354319 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:15.355070 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:15.355710 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.355729 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.356929 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.356933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.358194 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.358207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.358994 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.359002 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.359229 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.360012 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:15.360046 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:15.363685 (+ 3639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":286,"threads_started":1}
W20260928 14:51:15.365552 24437 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:38777 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.241.254:38777: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:15.373378 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:15.366144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:54335)
0928 14:51:15.366341 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:15.366361 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:15.366478 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:15.366784 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:15.366788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:15.367114 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:15.367260 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.367269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.368222 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.368227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.369139 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.369151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.369799 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.369804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.369961 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.370468 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:15.370485 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:15.373110 (+ 2625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260928 14:51:15.373533 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:15.366214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:54335 (local address 127.14.241.254:38777)
0928 14:51:15.366364 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:15.366369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:15.366423 (+ 54us) server_negotiation.cc:407] Connection header received
0928 14:51:15.366586 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:15.366590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:15.366650 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:15.366769 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:15.367394 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.368100 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.369267 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.369667 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.369784 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.373307 (+ 3523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:54335: BlockingRecv error: recv got EOF from 127.14.241.194:54335 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:15.383149 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:15.374137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:40069)
0928 14:51:15.374358 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:15.374375 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:15.374473 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:15.374756 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:15.374760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:15.375043 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:15.375178 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.375185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.376555 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.376561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.378179 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.378195 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.378863 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.378868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.379053 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.379587 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:15.379609 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:15.382834 (+ 3225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133}
W20260928 14:51:15.383209 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:15.374185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:40069 (local address 127.14.241.254:38777)
0928 14:51:15.374339 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:15.374344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:15.374420 (+ 76us) server_negotiation.cc:407] Connection header received
0928 14:51:15.374580 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:15.374585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:15.374647 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:15.374722 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:15.375314 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.376275 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.378331 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.378709 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.378837 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.383018 (+ 4181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:40069: BlockingRecv error: recv got EOF from 127.14.241.194:40069 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260928 14:51:15.383671 24437 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:51:15.670998 24440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:15.671330 24440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:15.671471 24440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:15.678850 24440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:15.678960 24440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:15.679103 24440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:51:15.688862 24440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:37647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/krb5kdc/oryx_127.14.241.195.keytab
--principal=oryx/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:32781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=39635
--tserver_master_addrs=127.14.241.254:38777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:15.690124 24440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:15.691382 24440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:15.703677 24448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:15.703701 24446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:15.703701 24445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:15.704524 24440 server_base.cc:1034] running on GCE node
I20260928 14:51:15.705070 24440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:15.705907 24440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:15.707095 24440 hybrid_clock.cc:648] HybridClock initialized: now 1790607075707036 us; error 50 us; skew 500 ppm
I20260928 14:51:15.707379 24440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607075, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:15.710377 24440 init.cc:377] Logged in from keytab as oryx/127.14.241.195@KRBTEST.COM (short username oryx)
I20260928 14:51:15.711721 24440 webserver.cc:460] Webserver started at http://127.14.241.195:39635/ using document root <none> and password file <none>
I20260928 14:51:15.712383 24440 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:15.712502 24440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:15.717633 24440 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:51:15.720321 24455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:15.721719 24440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:51:15.721915 24440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "f8536995377c40b4bc49c13961b0d08b"
format_stamp: "Formatted at 2026-09-28 14:51:13 on dist-test-slave-vk40"
server_key: "d8fccab7dce3789606aede299982137a"
server_key_iv: "c47859c4347d756d41208dcec50a5bca"
server_key_version: "encryptionkey@0"
I20260928 14:51:15.722867 24440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:15.738421 24440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:15.742622 24440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:15.742906 24440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:15.743818 24440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:15.745131 24440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:15.745225 24440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:15.745327 24440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:15.745399 24440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:15.774289 24440 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:32781
I20260928 14:51:15.774338 24567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:32781 every 8 connection(s)
I20260928 14:51:15.775655 24440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:51:15.783229 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 24440
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 14:51:15.792126 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:15.778520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:58647 (local address 127.14.241.254:38777)
0928 14:51:15.778689 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:15.778693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:15.779858 (+ 1165us) server_negotiation.cc:407] Connection header received
0928 14:51:15.781680 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:15.781685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:15.781759 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:15.781850 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:15.783441 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.784247 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.786507 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.787063 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.787175 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.791841 (+ 4666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:58647: BlockingRecv error: recv got EOF from 127.14.241.195:58647 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:51:15.792870 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:15.779045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:58647)
0928 14:51:15.779702 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:15.779739 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:15.781405 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:15.782077 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:15.782089 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:15.782629 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:15.783277 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.783296 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.784461 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.784467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.786384 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.786398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.787263 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.787268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.787478 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.788097 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:15.788120 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:15.791627 (+ 3507us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":253,"threads_started":1}
W20260928 14:51:15.793517 24568 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:38777 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.241.254:38777: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
I20260928 14:51:15.798988 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:15.784200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46380 (local address 127.14.241.254:38777)
0928 14:51:15.784857 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:15.784862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:15.784878 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:51:15.784938 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:15.784942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:15.784994 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:15.785066 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:15.786817 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.787727 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.789019 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.789437 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.789555 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.791997 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:15.792006 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:15.792023 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:15.792059 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:15.795691 (+ 3632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:15.796368 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:15.796377 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:15.796385 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:15.796628 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:15.796991 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:15.796996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:15.796999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:15.797613 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:15.797914 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:15.798290 (+ 376us) server_negotiation.cc:299] Negotiation successful
0928 14:51:15.798584 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":439,"threads_started":1}
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:15.802803 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:15.794117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:42291)
0928 14:51:15.794361 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:15.794380 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:15.794476 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:15.794807 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:15.794812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:15.795185 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:15.795364 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.795375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.796414 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.796419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.797643 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.797657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.798624 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.798630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.798810 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.799451 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:15.799473 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:15.802610 (+ 3137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136}
W20260928 14:51:15.803073 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:15.794238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:42291 (local address 127.14.241.254:38777)
0928 14:51:15.794404 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:15.794411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:15.794425 (+ 14us) server_negotiation.cc:407] Connection header received
0928 14:51:15.794588 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:15.794596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:15.794679 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:15.794756 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:15.795531 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.796259 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.797844 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.798477 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.798595 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.802830 (+ 4235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:42291: BlockingRecv error: recv got EOF from 127.14.241.195:42291 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:15.810448 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:15.803577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:39987)
0928 14:51:15.803718 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:15.803766 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:15.803857 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:15.804091 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:15.804094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:15.804401 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:15.804547 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.804556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.805530 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.805533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.806419 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.806429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.807085 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.807089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.807238 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.807777 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:15.807794 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:15.810241 (+ 2447us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 14:51:15.810730 24568 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:51:15.810885 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:15.803666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:39987 (local address 127.14.241.254:38777)
0928 14:51:15.803897 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:15.803900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:15.803915 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:51:15.803953 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:15.803956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:15.804008 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:15.804078 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:15.804675 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.805419 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.806520 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.806946 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.807035 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.810459 (+ 3424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:39987: BlockingRecv error: recv got EOF from 127.14.241.195:39987 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:15 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:15.948314 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:15.940125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:55559)
0928 14:51:15.940635 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:15.940657 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:15.940782 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:15.941108 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:15.941112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:15.941611 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:15.941844 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.941856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.942796 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.942799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.943755 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:15.943767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.944479 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.944484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:15.944613 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.945194 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:15.945212 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:15.947990 (+ 2778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":272,"threads_started":1}
W20260928 14:51:15.948479 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:15.940236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:55559 (local address 127.14.241.254:38777)
0928 14:51:15.940473 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:15.940477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:15.940725 (+ 248us) server_negotiation.cc:407] Connection header received
0928 14:51:15.940875 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:15.940878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:15.940938 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:15.941021 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:15.941980 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.942677 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.943897 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:15.944342 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:15.944433 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:15.948226 (+ 3793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:55559: BlockingRecv error: recv got EOF from 127.14.241.193:55559 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
Sep 28 14:51:16 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:16 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:16.391798 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:16.384250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:55605)
0928 14:51:16.384704 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:16.384720 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:16.384839 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:16.385106 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:16.385109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:16.385539 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:16.385718 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:16.385727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.386636 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.386640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:16.387602 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:16.387613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.388337 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.388344 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:16.388470 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:16.389007 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:16.389023 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:16.391572 (+ 2549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":210,"threads_started":1}
W20260928 14:51:16.392078 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:16.384386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:55605 (local address 127.14.241.254:38777)
0928 14:51:16.384610 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:16.384615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:16.384786 (+ 171us) server_negotiation.cc:407] Connection header received
0928 14:51:16.384934 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:16.384938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:16.385004 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:16.385081 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:16.385822 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.386534 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.387744 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.388159 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.388262 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:16.391830 (+ 3568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:55605: BlockingRecv error: recv got EOF from 127.14.241.194:55605 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 28 14:51:16 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:16 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:16.826745 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:16.811384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:60459)
0928 14:51:16.811918 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:16.811935 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:16.812093 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:16.812458 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:16.812463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:16.812990 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:16.813226 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:16.813241 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.814447 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.814451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:16.815387 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:16.815398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.816054 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.816058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:16.816175 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:16.823196 (+ 7021us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:16.823218 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:16.826480 (+ 3262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":277,"threads_started":1}
W20260928 14:51:16.826994 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:16.811521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:60459 (local address 127.14.241.254:38777)
0928 14:51:16.811749 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:16.811755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:16.812023 (+ 268us) server_negotiation.cc:407] Connection header received
0928 14:51:16.812214 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:16.812219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:16.812325 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:16.812418 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:16.813368 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.814320 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.815524 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.815918 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.816009 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:16.826743 (+ 10734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:60459: BlockingRecv error: recv got EOF from 127.14.241.195:60459 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 28 14:51:16 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:16 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:16.957270 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:16.949273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:54815)
0928 14:51:16.949789 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:16.949809 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:16.949952 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:16.950275 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:16.950280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:16.950694 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:16.950875 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:16.950885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.951931 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.951937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:16.952888 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:16.952901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.953595 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.953601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:16.953751 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:16.954277 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:16.954295 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:16.957037 (+ 2742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":278,"threads_started":1}
W20260928 14:51:16.957489 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:16.949367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:54815 (local address 127.14.241.254:38777)
0928 14:51:16.949568 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:16.949574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:16.949895 (+ 321us) server_negotiation.cc:407] Connection header received
0928 14:51:16.950069 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:16.950075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:16.950142 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:16.950222 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:16.951022 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.951765 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.953027 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:16.953445 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:16.953567 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:16.957236 (+ 3669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:54815: BlockingRecv error: recv got EOF from 127.14.241.193:54815 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 28 14:51:17 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:17 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:17.400658 24577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:17.392869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:33769)
0928 14:51:17.393384 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:17.393401 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:17.393514 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:17.393828 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:17.393832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:17.394224 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:17.394413 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:17.394424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.395358 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.395362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:17.396245 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:17.396256 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.397019 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.397025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:17.397179 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:17.397700 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:17.397716 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:17.400352 (+ 2636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":263,"threads_started":1}
W20260928 14:51:17.400803 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:17.392949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:33769 (local address 127.14.241.254:38777)
0928 14:51:17.393115 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:17.393120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:17.393467 (+ 347us) server_negotiation.cc:407] Connection header received
0928 14:51:17.393626 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:17.393630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:17.393693 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:17.393805 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:17.394533 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.395250 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.396425 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.396873 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.396983 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:17.400593 (+ 3610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:33769: BlockingRecv error: recv got EOF from 127.14.241.194:33769 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 28 14:51:17 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:17 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:17.835016 24578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:17.827593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:50285)
0928 14:51:17.828086 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:17.828104 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:17.828215 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:17.828549 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:17.828552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:17.828923 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:17.829117 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:17.829130 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.830070 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.830074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:17.830951 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:17.830962 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.831614 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.831618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:17.831731 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:17.832236 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:17.832253 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:17.834778 (+ 2525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":257,"threads_started":1}
W20260928 14:51:17.835208 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:17.827746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:50285 (local address 127.14.241.254:38777)
0928 14:51:17.827921 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:17.827926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:17.828197 (+ 271us) server_negotiation.cc:407] Connection header received
0928 14:51:17.828331 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:17.828336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:17.828422 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:17.828509 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:17.829231 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.829959 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.831079 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.831501 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.831591 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:17.834982 (+ 3391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:50285: BlockingRecv error: recv got EOF from 127.14.241.195:50285 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 28 14:51:17 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:17 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:17.966244 24579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:17.958261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:59661)
0928 14:51:17.958752 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:17.958777 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:17.958915 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:17.959252 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:17.959257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:17.959613 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:17.959770 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:17.959780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.960870 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.960876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:17.961757 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:17.961769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.962412 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.962418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:17.962582 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:17.963113 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:17.963131 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:17.965897 (+ 2766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":260,"threads_started":1}
W20260928 14:51:17.966378 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:17.958291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:59661 (local address 127.14.241.254:38777)
0928 14:51:17.958508 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:17.958514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:17.958863 (+ 349us) server_negotiation.cc:407] Connection header received
0928 14:51:17.959026 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:17.959032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:17.959106 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:17.959192 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:17.959918 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.960717 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.961893 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:17.962279 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:17.962390 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:17.966128 (+ 3738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:59661: BlockingRecv error: recv got EOF from 127.14.241.193:59661 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 28 14:51:18 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:18 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:18.409762 24580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:18.401654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:50639)
0928 14:51:18.402322 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:18.402343 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:18.402477 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:18.402816 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:18.402821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:18.403179 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:18.403348 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:18.403357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.404421 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.404427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:18.405338 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:18.405350 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.406020 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.406026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:18.406190 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:18.406735 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:18.406751 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:18.409409 (+ 2658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":236,"threads_started":1}
W20260928 14:51:18.409875 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:18.401778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:50639 (local address 127.14.241.254:38777)
0928 14:51:18.401988 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:18.401995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:18.402430 (+ 435us) server_negotiation.cc:407] Connection header received
0928 14:51:18.402598 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:18.402604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:18.402674 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:18.402757 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:18.403492 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.404273 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.405476 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.405888 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.406004 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:18.409636 (+ 3632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:50639: BlockingRecv error: recv got EOF from 127.14.241.194:50639 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 28 14:51:18 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:18 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:18.843233 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:18.835824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:48281)
0928 14:51:18.836258 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:18.836275 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:18.836428 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:18.836686 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:18.836689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:18.837017 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:18.837180 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:18.837188 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.838144 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.838148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:18.839026 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:18.839037 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.839634 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.839638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:18.839751 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:18.840274 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:18.840321 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:18.842979 (+ 2658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":217,"threads_started":1}
W20260928 14:51:18.843367 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:18.835962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:48281 (local address 127.14.241.254:38777)
0928 14:51:18.836174 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:18.836179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:18.836377 (+ 198us) server_negotiation.cc:407] Connection header received
0928 14:51:18.836519 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:18.836522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:18.836583 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:18.836661 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:18.837291 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.838032 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.839136 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.839527 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.839615 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:18.843175 (+ 3560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:48281: BlockingRecv error: recv got EOF from 127.14.241.195:48281 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 28 14:51:18 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:18 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:18.974849 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:18.967193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:46805)
0928 14:51:18.967687 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:18.967706 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:18.967816 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:18.968171 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:18.968175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:18.968556 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:18.968705 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:18.968714 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.969685 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.969689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:18.970696 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:18.970708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.971375 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.971379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:18.971495 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:18.972031 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:18.972047 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:18.974618 (+ 2571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":274,"threads_started":1}
W20260928 14:51:18.975113 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:18.967247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:46805 (local address 127.14.241.254:38777)
0928 14:51:18.967454 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:18.967458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:18.967778 (+ 320us) server_negotiation.cc:407] Connection header received
0928 14:51:18.967951 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:18.967955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:18.968019 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:18.968100 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:18.968824 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.969561 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.970837 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:18.971256 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:18.971346 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:18.974848 (+ 3502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:46805: BlockingRecv error: recv got EOF from 127.14.241.193:46805 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 28 14:51:19 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:19 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:19.419386 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:19.410813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:45157)
0928 14:51:19.411377 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:19.411392 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:19.411543 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:19.411875 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:19.411879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:19.412315 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:19.412477 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:19.412487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.413474 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.413478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:19.414616 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:19.414630 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.415471 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.415476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:19.415619 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:19.416259 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:19.416321 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:19.419128 (+ 2807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":311,"threads_started":1}
W20260928 14:51:19.419551 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:19.410947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:45157 (local address 127.14.241.254:38777)
0928 14:51:19.411183 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:19.411190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:19.411485 (+ 295us) server_negotiation.cc:407] Connection header received
0928 14:51:19.411671 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:19.411676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:19.411741 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:19.411825 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:19.412628 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.413340 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.414762 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.415309 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.415427 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:19.419334 (+ 3907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:45157: BlockingRecv error: recv got EOF from 127.14.241.194:45157 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 28 14:51:19 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:19 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 14:51:19.852664 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:19.844323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:58705 (local address 127.14.241.254:38777)
0928 14:51:19.844559 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:19.844565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:19.844765 (+ 200us) server_negotiation.cc:407] Connection header received
0928 14:51:19.844901 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:19.844904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:19.844974 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:19.845060 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:19.845717 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.846516 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.847739 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.848151 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.848243 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:19.852378 (+ 4135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:58705: BlockingRecv error: recv got EOF from 127.14.241.195:58705 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 14:51:19.853225 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:19.844203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:58705)
0928 14:51:19.844650 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:19.844666 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:19.844784 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:19.845099 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:19.845102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:19.845432 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:19.845577 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:19.845585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.846643 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.846647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:19.847617 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:19.847628 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.848261 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.848265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:19.848423 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:19.849006 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:19.849024 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:19.852920 (+ 3896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":254,"threads_started":1}
Sep 28 14:51:19 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:19 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:19.983834 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:19.975804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:37443)
0928 14:51:19.976344 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:19.976364 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:19.976474 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:19.976773 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:19.976777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:19.977166 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:19.977328 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:19.977337 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.978358 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.978361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:19.979275 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:19.979287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.979878 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.979881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:19.980004 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:19.980558 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:19.980574 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:19.983552 (+ 2978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":264,"threads_started":1}
W20260928 14:51:19.983954 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:19.975886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:37443 (local address 127.14.241.254:38777)
0928 14:51:19.976062 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:19.976066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:19.976428 (+ 362us) server_negotiation.cc:407] Connection header received
0928 14:51:19.976579 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:19.976583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:19.976661 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:19.976729 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:19.977479 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.978249 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.979380 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:19.979778 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:19.979862 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:19.983770 (+ 3908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:37443: BlockingRecv error: recv got EOF from 127.14.241.193:37443 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 28 14:51:20 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:20 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:20.429098 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:20.420318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:49287)
0928 14:51:20.420814 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:20.420829 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:20.420945 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:20.421259 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:20.421262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:20.421641 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:20.421807 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:20.421816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.423200 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.423204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:20.424119 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:20.424132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.425116 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.425120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:20.425240 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:20.425871 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:20.425889 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:20.428773 (+ 2884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":233,"threads_started":1}
W20260928 14:51:20.429242 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:20.420510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:49287 (local address 127.14.241.254:38777)
0928 14:51:20.420703 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:20.420709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:20.420895 (+ 186us) server_negotiation.cc:407] Connection header received
0928 14:51:20.421046 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:20.421052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:20.421131 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:20.421227 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:20.421926 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.423081 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.424253 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.424997 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.425112 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:20.428970 (+ 3858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:49287: BlockingRecv error: recv got EOF from 127.14.241.194:49287 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 28 14:51:20 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:20 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:20.862373 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:20.854137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:53781)
0928 14:51:20.854662 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:20.854679 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:20.854818 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:20.855134 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:20.855138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:20.855583 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:20.855750 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:20.855761 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.856820 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.856826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:20.857762 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:20.857776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.858463 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.858469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:20.858627 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:20.859208 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:20.859226 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:20.862039 (+ 2813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":261,"threads_started":1}
W20260928 14:51:20.862531 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:20.854289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:53781 (local address 127.14.241.254:38777)
0928 14:51:20.854517 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:20.854524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:20.854768 (+ 244us) server_negotiation.cc:407] Connection header received
0928 14:51:20.854938 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:20.854943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:20.855006 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:20.855089 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:20.855904 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.856680 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.857904 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.858322 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.858431 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:20.862257 (+ 3826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:53781: BlockingRecv error: recv got EOF from 127.14.241.195:53781 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 28 14:51:20 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:20 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:20.992349 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:20.984801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:44947)
0928 14:51:20.985353 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:20.985376 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:20.985487 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:20.985759 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:20.985762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:20.986132 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:20.986283 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:20.986292 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.987276 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.987279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:20.988170 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:20.988181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.988864 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.988869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:20.988990 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:20.989529 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:20.989546 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:20.992082 (+ 2536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":303,"threads_started":1}
W20260928 14:51:20.992515 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:20.984810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:44947 (local address 127.14.241.254:38777)
0928 14:51:20.985054 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:20.985061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:20.985448 (+ 387us) server_negotiation.cc:407] Connection header received
0928 14:51:20.985580 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:20.985584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:20.985647 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:20.985727 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:20.986416 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.987173 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.988274 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:20.988712 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:20.988802 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:20.992305 (+ 3503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:44947: BlockingRecv error: recv got EOF from 127.14.241.193:44947 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 28 14:51:21 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:21 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:21.438359 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:21.430072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:33601)
0928 14:51:21.430839 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:21.430855 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:21.431015 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:21.431339 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:21.431344 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:21.431745 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:21.431913 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:21.431923 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.433000 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.433004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:21.433937 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:21.433951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.434654 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.434660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:21.434821 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:21.435390 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:21.435410 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:21.438061 (+ 2651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":529,"threads_started":1}
W20260928 14:51:21.438591 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:21.430157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:33601 (local address 127.14.241.254:38777)
0928 14:51:21.430347 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:21.430354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:21.430940 (+ 586us) server_negotiation.cc:407] Connection header received
0928 14:51:21.431137 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:21.431143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:21.431205 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:21.431285 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:21.432066 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.432836 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.434089 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.434504 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.434620 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:21.438341 (+ 3721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:33601: BlockingRecv error: recv got EOF from 127.14.241.194:33601 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 28 14:51:21 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:21 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:21.875699 24590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:21.863460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:49919)
0928 14:51:21.864007 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:21.864028 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:21.864149 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:21.864524 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:21.864529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:21.864967 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:21.865187 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:21.865202 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.866315 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.866319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:21.867234 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:21.867246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.867981 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.867985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:21.868099 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:21.872524 (+ 4425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:21.872543 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:21.875453 (+ 2910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":301,"threads_started":1}
W20260928 14:51:21.876080 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:21.863580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:49919 (local address 127.14.241.254:38777)
0928 14:51:21.863808 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:21.863814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:21.864119 (+ 305us) server_negotiation.cc:407] Connection header received
0928 14:51:21.864269 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:21.864273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:21.864378 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:21.864470 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:21.865316 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.866174 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.867395 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.867850 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.867952 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:21.875700 (+ 7748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:49919: BlockingRecv error: recv got EOF from 127.14.241.195:49919 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 28 14:51:21 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:21 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:22.001580 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:21.993319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:53821)
0928 14:51:21.993837 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:21.993856 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:21.993976 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:21.994246 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:21.994249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:21.994688 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:21.994858 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:21.994868 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.995838 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.995842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:21.996825 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:21.996841 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.997692 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.997697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:21.997864 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:21.998475 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:21.998492 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:22.001220 (+ 2728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":224,"threads_started":1}
W20260928 14:51:22.001655 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:21.993423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:53821 (local address 127.14.241.254:38777)
0928 14:51:21.993625 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:21.993633 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:21.993929 (+ 296us) server_negotiation.cc:407] Connection header received
0928 14:51:21.994070 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:21.994073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:21.994136 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:21.994217 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:21.994994 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.995721 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.996974 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:21.997550 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:21.997671 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:22.001442 (+ 3771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:53821: BlockingRecv error: recv got EOF from 127.14.241.193:53821 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 28 14:51:22 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:22 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:22.447585 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:22.439312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:33831)
0928 14:51:22.439882 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:22.439900 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:22.440027 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:22.440382 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:22.440386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:22.440851 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:22.441025 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:22.441035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:22.442157 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:22.442162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:22.443228 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:22.443242 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:22.443999 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:22.444003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:22.444163 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:22.444792 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:22.444810 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:22.447344 (+ 2534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":244,"threads_started":1}
W20260928 14:51:22.447794 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:22.439393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:33831 (local address 127.14.241.254:38777)
0928 14:51:22.439621 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:22.439626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:22.439968 (+ 342us) server_negotiation.cc:407] Connection header received
0928 14:51:22.440129 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:22.440133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:22.440199 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:22.440310 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:22.441159 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:22.442008 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:22.443332 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:22.443839 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:22.443957 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:22.447559 (+ 3602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:33831: BlockingRecv error: recv got EOF from 127.14.241.194:33831 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Sep 28 14:51:22 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:22 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:22.884783 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:22.876767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:53643)
0928 14:51:22.877368 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:22.877389 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:22.877514 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:22.877866 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:22.877870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:22.878279 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:22.878451 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:22.878460 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:22.879454 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:22.879460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:22.880465 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:22.880480 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:22.881153 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:22.881157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:22.881271 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:22.881816 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:22.881832 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:22.884535 (+ 2703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":255,"threads_started":1}
W20260928 14:51:22.884903 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:22.876848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:53643 (local address 127.14.241.254:38777)
0928 14:51:22.877028 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:22.877034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:22.877466 (+ 432us) server_negotiation.cc:407] Connection header received
0928 14:51:22.877637 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:22.877641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:22.877707 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:22.877816 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:22.878594 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:22.879312 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:22.880601 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:22.881029 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:22.881125 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:22.884714 (+ 3589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:53643: BlockingRecv error: recv got EOF from 127.14.241.195:53643 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 28 14:51:23 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:23 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:23.010746 24594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:23.002490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:36459)
0928 14:51:23.002971 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:23.002990 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:23.003106 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:23.003397 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:23.003401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:23.003775 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:23.003942 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:23.003951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.005033 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.005037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:23.005942 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:23.005954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.006721 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.006725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:23.006833 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:23.007408 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:23.007425 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:23.010439 (+ 3014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":225,"threads_started":1}
W20260928 14:51:23.010898 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:23.002702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:36459 (local address 127.14.241.254:38777)
0928 14:51:23.002932 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:23.002936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:23.003092 (+ 156us) server_negotiation.cc:407] Connection header received
0928 14:51:23.003213 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:23.003217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:23.003292 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:23.003367 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:23.004081 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.004913 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.006082 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.006570 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.006699 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:23.010653 (+ 3954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:36459: BlockingRecv error: recv got EOF from 127.14.241.193:36459 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 28 14:51:23 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:23 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:23.456578 24595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:23.448502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:43189)
0928 14:51:23.449010 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:23.449026 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:23.449137 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:23.449465 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:23.449469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:23.449877 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:23.450035 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:23.450044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.451272 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.451276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:23.452204 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:23.452217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.452931 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.452936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:23.453052 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:23.453665 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:23.453683 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:23.456231 (+ 2548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":255,"threads_started":1}
W20260928 14:51:23.456681 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:23.448596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:43189 (local address 127.14.241.254:38777)
0928 14:51:23.448752 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:23.448756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:23.449089 (+ 333us) server_negotiation.cc:407] Connection header received
0928 14:51:23.449232 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:23.449235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:23.449297 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:23.449383 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:23.450197 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.451117 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.452367 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.452784 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.452877 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:23.456478 (+ 3601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:43189: BlockingRecv error: recv got EOF from 127.14.241.194:43189 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 28 14:51:23 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:23 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:23.893883 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:23.885661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:35571)
0928 14:51:23.886123 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:23.886139 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:23.886257 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:23.886536 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:23.886540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:23.886931 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:23.887085 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:23.887093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.888053 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.888056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:23.889006 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:23.889019 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.889669 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.889673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:23.889778 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:23.890392 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:23.890417 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:23.893531 (+ 3114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":231,"threads_started":1}
W20260928 14:51:23.894043 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:23.885765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:35571 (local address 127.14.241.254:38777)
0928 14:51:23.886030 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:23.886035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:23.886211 (+ 176us) server_negotiation.cc:407] Connection header received
0928 14:51:23.886352 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:23.886356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:23.886425 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:23.886507 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:23.887207 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.887936 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.889120 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:23.889533 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:23.889622 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:23.893780 (+ 4158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:35571: BlockingRecv error: recv got EOF from 127.14.241.195:35571 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 28 14:51:24 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:24 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:24.020006 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:24.011788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:60269)
0928 14:51:24.012300 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:24.012326 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:24.012444 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:24.012854 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:24.012859 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:24.013325 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:24.013497 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:24.013507 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.014519 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.014523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:24.015465 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:24.015476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.016091 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.016095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:24.016212 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:24.016826 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:24.016862 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:24.019763 (+ 2901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":242,"threads_started":1}
W20260928 14:51:24.020186 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:24.011835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:60269 (local address 127.14.241.254:38777)
0928 14:51:24.012072 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:24.012078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:24.012414 (+ 336us) server_negotiation.cc:407] Connection header received
0928 14:51:24.012567 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:24.012571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:24.012654 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:24.012768 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:24.013623 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.014393 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.015574 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.015983 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.016100 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:24.019969 (+ 3869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:60269: BlockingRecv error: recv got EOF from 127.14.241.193:60269 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 28 14:51:24 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:24 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:24.466089 24598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:24.457686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:39443)
0928 14:51:24.458192 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:24.458210 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:24.458342 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:24.458671 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:24.458674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:24.459075 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:24.459243 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:24.459252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.460323 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.460329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:24.461583 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:24.461599 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.462414 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.462419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:24.462548 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:24.463144 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:24.463161 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:24.465846 (+ 2685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":254,"threads_started":1}
W20260928 14:51:24.466275 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:24.457800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:39443 (local address 127.14.241.254:38777)
0928 14:51:24.457998 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:24.458003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:24.458334 (+ 331us) server_negotiation.cc:407] Connection header received
0928 14:51:24.458453 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:24.458456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:24.458523 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:24.458614 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:24.459392 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.460144 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.461727 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.462235 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.462333 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:24.466044 (+ 3711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:39443: BlockingRecv error: recv got EOF from 127.14.241.194:39443 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 28 14:51:24 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:24 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:24.902917 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:24.894907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:47525)
0928 14:51:24.895424 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:24.895444 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:24.895573 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:24.895945 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:24.895950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:24.896383 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:24.896558 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:24.896568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.897528 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.897532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:24.898470 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:24.898482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.899095 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.899099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:24.899211 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:24.899758 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:24.899776 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:24.902577 (+ 2801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":244,"threads_started":1}
W20260928 14:51:24.903012 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:24.894972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:47525 (local address 127.14.241.254:38777)
0928 14:51:24.895154 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:24.895159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:24.895521 (+ 362us) server_negotiation.cc:407] Connection header received
0928 14:51:24.895698 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:24.895703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:24.895778 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:24.895872 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:24.896684 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.897398 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.898578 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:24.898988 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:24.899074 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:24.902796 (+ 3722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:47525: BlockingRecv error: recv got EOF from 127.14.241.195:47525 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 28 14:51:25 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:25 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:25.028868 24600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:25.020870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:58329)
0928 14:51:25.021354 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:25.021374 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:25.021501 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:25.021760 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:25.021763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:25.022119 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:25.022268 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:25.022277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.023269 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.023274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:25.024237 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:25.024252 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.025056 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.025060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:25.025185 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:25.025815 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:25.025837 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:25.028646 (+ 2809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":260,"threads_started":1}
W20260928 14:51:25.029058 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:25.021011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:58329 (local address 127.14.241.254:38777)
0928 14:51:25.021235 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:25.021241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:25.021457 (+ 216us) server_negotiation.cc:407] Connection header received
0928 14:51:25.021590 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:25.021594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:25.021655 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:25.021731 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:25.022379 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.023112 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.024423 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.024898 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.024998 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:25.028811 (+ 3813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:58329: BlockingRecv error: recv got EOF from 127.14.241.193:58329 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 28 14:51:25 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:25 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:25.475131 24601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:25.467014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:59557)
0928 14:51:25.467485 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:25.467501 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:25.467626 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:25.467953 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:25.467958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:25.468362 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:25.468515 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:25.468524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.469513 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.469517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:25.470439 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:25.470451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.471110 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.471116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:25.471264 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:25.471787 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:25.471803 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:25.474790 (+ 2987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":261,"threads_started":1}
W20260928 14:51:25.475232 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:25.467094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:59557 (local address 127.14.241.254:38777)
0928 14:51:25.467273 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:25.467279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:25.467584 (+ 305us) server_negotiation.cc:407] Connection header received
0928 14:51:25.467748 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:25.467752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:25.467813 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:25.467893 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:25.468652 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.469385 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.470571 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.470976 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.471080 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:25.475012 (+ 3932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:59557: BlockingRecv error: recv got EOF from 127.14.241.194:59557 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 28 14:51:25 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:25 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:25.911784 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:25.903819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:46243)
0928 14:51:25.904353 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:25.904371 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:25.904526 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:25.904852 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:25.904856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:25.905223 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:25.905376 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:25.905385 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.906415 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.906420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:25.907347 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:25.907359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.908068 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.908073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:25.908232 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:25.908788 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:25.908805 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:25.911470 (+ 2665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":250,"threads_started":1}
W20260928 14:51:25.911921 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:25.904038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:46243 (local address 127.14.241.254:38777)
0928 14:51:25.904314 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:25.904321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:25.904472 (+ 151us) server_negotiation.cc:407] Connection header received
0928 14:51:25.904655 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:25.904661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:25.904728 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:25.904809 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:25.905529 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.906268 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.907486 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:25.907923 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:25.908033 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:25.911680 (+ 3647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:46243: BlockingRecv error: recv got EOF from 127.14.241.195:46243 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Sep 28 14:51:26 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:26 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:26.037875 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:26.029722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:38405)
0928 14:51:26.030195 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:26.030215 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:26.030355 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:26.030692 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:26.030697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:26.031102 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:26.031278 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:26.031287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.032325 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.032331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:26.033250 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:26.033263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.034000 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.034006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:26.034168 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:26.034736 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:26.034752 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:26.037621 (+ 2869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":239,"threads_started":1}
W20260928 14:51:26.038038 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:26.029810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:38405 (local address 127.14.241.254:38777)
0928 14:51:26.030052 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:26.030058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:26.030304 (+ 246us) server_negotiation.cc:407] Connection header received
0928 14:51:26.030472 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:26.030478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:26.030557 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:26.030639 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:26.031426 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.032161 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.033391 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.033836 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.033957 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:26.037805 (+ 3848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:38405: BlockingRecv error: recv got EOF from 127.14.241.193:38405 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 28 14:51:26 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:26 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:26.484346 24604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:26.476041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:50605)
0928 14:51:26.476540 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:26.476557 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:26.476673 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:26.476967 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:26.476969 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:26.477375 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:26.477566 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:26.477575 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.478538 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.478542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:26.479633 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:26.479645 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.480475 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.480480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:26.480613 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:26.481187 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:26.481204 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:26.484050 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":287,"threads_started":1}
W20260928 14:51:26.484514 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:26.476143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:50605 (local address 127.14.241.254:38777)
0928 14:51:26.476381 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:26.476386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:26.476631 (+ 245us) server_negotiation.cc:407] Connection header received
0928 14:51:26.476772 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:26.476776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:26.476841 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:26.476933 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:26.477668 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.478423 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.479756 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.480335 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.480449 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:26.484256 (+ 3807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:50605: BlockingRecv error: recv got EOF from 127.14.241.194:50605 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 28 14:51:26 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:26 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:26.925024 24605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:26.912698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:33299)
0928 14:51:26.913162 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:26.913180 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:26.913295 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:26.913556 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:26.913560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:26.913956 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:26.914132 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:26.914140 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.915213 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.915217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:26.916346 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:26.916361 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.917059 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.917066 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:26.917239 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:26.921750 (+ 4511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:26.921777 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:26.924773 (+ 2996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":238,"threads_started":1}
W20260928 14:51:26.925168 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:26.912761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:33299 (local address 127.14.241.254:38777)
0928 14:51:26.912922 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:26.912926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:26.913243 (+ 317us) server_negotiation.cc:407] Connection header received
0928 14:51:26.913381 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:26.913384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:26.913445 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:26.913524 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:26.914259 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.915066 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.916495 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:26.916908 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:26.917015 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:26.924968 (+ 7953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:33299: BlockingRecv error: recv got EOF from 127.14.241.195:33299 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 28 14:51:27 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:27 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:27.046970 24606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:27.038763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:40797)
0928 14:51:27.039206 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:27.039226 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:27.039359 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:27.039638 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:27.039641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:27.040001 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:27.040164 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:27.040172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.041368 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.041372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:27.042381 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:27.042394 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.043266 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.043271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:27.043449 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:27.044058 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:27.044075 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:27.046723 (+ 2648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":245,"threads_started":1}
W20260928 14:51:27.047127 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:27.038885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:40797 (local address 127.14.241.254:38777)
0928 14:51:27.039144 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:27.039150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:27.039299 (+ 149us) server_negotiation.cc:407] Connection header received
0928 14:51:27.039449 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:27.039452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:27.039515 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:27.039598 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:27.040314 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.041212 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.042541 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.043130 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.043238 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:27.046917 (+ 3679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:40797: BlockingRecv error: recv got EOF from 127.14.241.193:40797 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Sep 28 14:51:27 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:27 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:27.493269 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:27.485229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:43897)
0928 14:51:27.485671 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:27.485687 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:27.485804 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:27.486089 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:27.486092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:27.486458 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:27.486613 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:27.486622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.487660 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.487666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:27.488892 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:27.488907 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.489589 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.489595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:27.489774 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:27.490307 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:27.490324 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:27.492974 (+ 2650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":246,"threads_started":1}
W20260928 14:51:27.493427 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:27.485287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:43897 (local address 127.14.241.254:38777)
0928 14:51:27.485514 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:27.485520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:27.485761 (+ 241us) server_negotiation.cc:407] Connection header received
0928 14:51:27.485901 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:27.485905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:27.485968 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:27.486046 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:27.486734 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.487520 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.489027 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.489451 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.489556 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:27.493211 (+ 3655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:43897: BlockingRecv error: recv got EOF from 127.14.241.194:43897 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 28 14:51:27 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:27 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:27.934475 24608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:27.925928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:51391)
0928 14:51:27.926390 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:27.926407 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:27.926560 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:27.926928 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:27.926931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:27.927411 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:27.927594 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:27.927605 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.928973 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.928977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:27.929999 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:27.930016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.930896 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.930912 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:27.931045 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:27.931635 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:27.931656 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:27.934245 (+ 2589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":251,"threads_started":1}
W20260928 14:51:27.934706 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:27.925999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:51391 (local address 127.14.241.254:38777)
0928 14:51:27.926148 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:27.926152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:27.926476 (+ 324us) server_negotiation.cc:407] Connection header received
0928 14:51:27.926656 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:27.926669 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:27.926754 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:27.926892 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:27.927755 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.928803 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.930151 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:27.930780 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:27.930880 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:27.934473 (+ 3593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:51391: BlockingRecv error: recv got EOF from 127.14.241.195:51391 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 28 14:51:28 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:28 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:28.057948 24609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:28.048062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:42693)
0928 14:51:28.048625 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:28.048645 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:28.048807 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:28.049139 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:28.049144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:28.049558 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:28.049746 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:28.049756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.050855 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.050862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:28.052474 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:28.052491 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.053313 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.053320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:28.053520 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:28.054176 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:28.054199 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:28.057609 (+ 3410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":291,"threads_started":1}
W20260928 14:51:28.058066 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:28.048163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:42693 (local address 127.14.241.254:38777)
0928 14:51:28.048390 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:28.048395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:28.048739 (+ 344us) server_negotiation.cc:407] Connection header received
0928 14:51:28.048938 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:28.048942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:28.049009 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:28.049095 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:28.049904 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.050690 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.052671 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.053125 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.053224 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:28.057833 (+ 4609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:42693: BlockingRecv error: recv got EOF from 127.14.241.193:42693 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 28 14:51:28 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:28 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:28.502179 24610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:28.494090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:36865)
0928 14:51:28.494590 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:28.494606 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:28.494739 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:28.495154 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:28.495160 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:28.495692 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:28.495873 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:28.495882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.496863 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.496867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:28.497773 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:28.497784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.498408 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.498432 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:28.498592 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:28.499114 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:28.499130 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:28.501873 (+ 2743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":276,"threads_started":1}
W20260928 14:51:28.502331 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:28.494259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:36865 (local address 127.14.241.254:38777)
0928 14:51:28.494446 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:28.494452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:28.494683 (+ 231us) server_negotiation.cc:407] Connection header received
0928 14:51:28.494902 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:28.494907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:28.494977 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:28.495070 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:28.496005 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.496754 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.497886 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.498276 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.498360 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:28.502105 (+ 3745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:36865: BlockingRecv error: recv got EOF from 127.14.241.194:36865 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 28 14:51:28 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:28 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:28.942919 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:28.935366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:38439)
0928 14:51:28.935858 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:28.935875 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:28.935983 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:28.936243 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:28.936246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:28.936632 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:28.936792 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:28.936800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.937753 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.937757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:28.938711 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:28.938722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.939399 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.939404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:28.939541 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:28.940072 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:28.940088 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:28.942676 (+ 2588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":244,"threads_started":1}
W20260928 14:51:28.943071 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:28.935410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:38439 (local address 127.14.241.254:38777)
0928 14:51:28.935597 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:28.935604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:28.935948 (+ 344us) server_negotiation.cc:407] Connection header received
0928 14:51:28.936074 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:28.936078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:28.936139 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:28.936215 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:28.936932 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.937637 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.938816 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:28.939220 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:28.939306 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:28.942875 (+ 3569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:38439: BlockingRecv error: recv got EOF from 127.14.241.195:38439 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 28 14:51:29 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:29 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:29.066696 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:29.058942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:57821)
0928 14:51:29.059387 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:29.059408 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:29.059524 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:29.059823 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:29.059826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:29.060176 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:29.060369 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:29.060378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.061368 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.061373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:29.062301 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:29.062313 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.062942 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.062946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:29.063071 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:29.063601 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:29.063618 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:29.066412 (+ 2794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":215,"threads_started":1}
W20260928 14:51:29.066864 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:29.059090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:57821 (local address 127.14.241.254:38777)
0928 14:51:29.059407 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:29.059413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:29.059477 (+ 64us) server_negotiation.cc:407] Connection header received
0928 14:51:29.059614 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:29.059617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:29.059710 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:29.059790 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:29.060482 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.061228 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.062411 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.062834 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.062930 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:29.066623 (+ 3693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:57821: BlockingRecv error: recv got EOF from 127.14.241.193:57821 (error 108)
Metrics: {"server-negotiator.queue_time_us":173}
Sep 28 14:51:29 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:29 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:29.511147 24613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:29.503088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:48641)
0928 14:51:29.503546 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:29.503561 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:29.503689 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:29.504006 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:29.504011 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:29.504445 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:29.504625 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:29.504634 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.505657 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.505661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:29.506554 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:29.506567 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.507268 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.507274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:29.507440 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:29.508011 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:29.508027 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:29.510849 (+ 2822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":244,"threads_started":1}
W20260928 14:51:29.511291 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:29.503212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:48641 (local address 127.14.241.254:38777)
0928 14:51:29.503410 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:29.503416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:29.503637 (+ 221us) server_negotiation.cc:407] Connection header received
0928 14:51:29.503817 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:29.503821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:29.503883 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:29.503962 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:29.504776 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.505525 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.506700 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.507135 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.507240 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:29.511057 (+ 3817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:48641: BlockingRecv error: recv got EOF from 127.14.241.194:48641 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 28 14:51:29 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:29 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.195@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:29.951813 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:29.943726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.195:46467)
0928 14:51:29.944226 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:29.944243 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:29.944382 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:29.944758 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:29.944764 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:29.945246 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:29.945459 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:29.945469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.946514 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.946519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:29.947439 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:29.947451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.948159 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.948165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:29.948383 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:29.948910 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:29.948927 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:29.951507 (+ 2580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":256,"threads_started":1}
W20260928 14:51:29.952013 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:29.943817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:46467 (local address 127.14.241.254:38777)
0928 14:51:29.943991 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:29.943997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:29.944320 (+ 323us) server_negotiation.cc:407] Connection header received
0928 14:51:29.944502 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:29.944507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:29.944587 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:29.944700 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:29.945602 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.946324 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.947575 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:29.948022 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:29.948127 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:29.951732 (+ 3605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.195:46467: BlockingRecv error: recv got EOF from 127.14.241.195:46467 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 28 14:51:30 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:30 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.193@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:30.075795 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:30.067834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.193:41163)
0928 14:51:30.068248 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:30.068267 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:30.068417 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:30.068707 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:30.068709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:30.069102 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:30.069262 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:30.069271 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.070213 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.070216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:30.071161 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:30.071172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.072056 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.072062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:30.072224 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:30.072847 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:30.072870 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:30.075535 (+ 2665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":217,"threads_started":1}
W20260928 14:51:30.075953 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:30.067920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:41163 (local address 127.14.241.254:38777)
0928 14:51:30.068142 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:30.068148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:30.068373 (+ 225us) server_negotiation.cc:407] Connection header received
0928 14:51:30.068511 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:30.068515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:30.068597 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:30.068682 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:30.069369 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.070102 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.071314 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.071921 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.072033 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:30.075740 (+ 3707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.193:41163: BlockingRecv error: recv got EOF from 127.14.241.193:41163 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 28 14:51:30 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:51:30 dist-test-slave-vk40 krb5kdc[23571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.241.194@KRBTEST.COM for oryx/127.14.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:51:30.520440 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:30.512467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38777 (local address 127.14.241.194:36513)
0928 14:51:30.513004 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:30.513020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:30.513133 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:30.513470 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:30.513475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:30.513897 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:30.514122 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:30.514135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.515332 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.515336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:30.516250 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:30.516263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.516940 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.516944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:30.517053 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:30.517577 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:30.517593 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:30.520157 (+ 2564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.241.254:38777: Server oryx/127.14.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":277,"threads_started":1}
W20260928 14:51:30.520598 24307 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:30.512510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:36513 (local address 127.14.241.254:38777)
0928 14:51:30.512666 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:30.512671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:30.513083 (+ 412us) server_negotiation.cc:407] Connection header received
0928 14:51:30.513259 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:30.513264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:30.513342 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:30.513445 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:30.514258 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.515202 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.516377 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.516818 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.516915 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:30.520373 (+ 3458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.241.194:36513: BlockingRecv error: recv got EOF from 127.14.241.194:36513 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 14:51:30.790122 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 24158
I20260928 14:51:30.802071 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:30.791305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51370 (local address 127.14.241.193:43275)
0928 14:51:30.791791 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:30.791799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:30.791827 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:51:30.791886 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:30.791892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:30.792124 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:30.792274 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:30.793712 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.794793 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.795976 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.796580 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.796706 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:30.798997 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:30.799006 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:30.799021 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:30.799055 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:30.801563 (+ 2508us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.241.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0928 14:51:30.801831 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51370: Request ticket server kudu/127.14.241.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":222,"threads_started":1}
W20260928 14:51:30.802165 24618 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51370: Request ticket server kudu/127.14.241.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260928 14:51:30.802248 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 24158: Not authorized: Client connection negotiation failed: client connection to 127.14.241.193:43275: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.241.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260928 14:51:30.802449 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 24158
I20260928 14:51:30.816855 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 24309
I20260928 14:51:30.828547 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:30.817845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.14.241.194:39185)
0928 14:51:30.818414 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:30.818422 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:30.818450 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:51:30.818520 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:30.818527 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:30.818741 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:30.818920 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:30.820180 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.821286 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.822466 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.823016 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.823129 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:30.825501 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:30.825510 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:30.825525 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:30.825558 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:30.827978 (+ 2420us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.241.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0928 14:51:30.828255 (+ 277us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44396: Request ticket server kudu/127.14.241.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":266,"threads_started":1}
W20260928 14:51:30.828644 24619 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44396: Request ticket server kudu/127.14.241.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260928 14:51:30.828676 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 24309: Not authorized: Client connection negotiation failed: client connection to 127.14.241.194:39185: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.241.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260928 14:51:30.828845 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 24309
I20260928 14:51:30.842275 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 24440
I20260928 14:51:30.854033 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:30.843218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.14.241.195:32781)
0928 14:51:30.843754 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:30.843760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:30.843786 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:51:30.843868 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:30.843875 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:30.844097 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:30.844239 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:30.845559 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.846571 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.847734 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:30.848358 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:30.848467 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:30.850789 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:30.850797 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:30.850812 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:30.850871 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:30.853522 (+ 2651us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.241.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0928 14:51:30.853788 (+ 266us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60546: Request ticket server kudu/127.14.241.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":251,"threads_started":1}
W20260928 14:51:30.854130 24620 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60546: Request ticket server kudu/127.14.241.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260928 14:51:30.854269 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 24440: Not authorized: Client connection negotiation failed: client connection to 127.14.241.195:32781: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.241.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260928 14:51:30.854431 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 24440
I20260928 14:51:30.868166 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 24089
I20260928 14:51:30.869536 24151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:30.990684 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 24089
2026-09-28T14:51:31Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20893 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-28T14:51:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:51:31Z Disabled control of system clock
I20260928 14:51:31.061313 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:33321
--webserver_interface=127.14.241.254
--webserver_port=0
--builtin_ntp_servers=127.14.241.212:44223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:33321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 14:51:31.381623 24625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:31.381937 24625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:31.382014 24625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:31.389469 24625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:51:31.389593 24625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:31.389631 24625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:31.389691 24625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:51:31.389735 24625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:51:31.398939 24625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:33321
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:33321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:31.400194 24625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:31.401963 24625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:31.413281 24633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:31.413345 24630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:31.413345 24631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:31.413604 24625 server_base.cc:1034] running on GCE node
I20260928 14:51:31.414569 24625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:31.416013 24625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:31.417213 24625 hybrid_clock.cc:648] HybridClock initialized: now 1790607091417167 us; error 35 us; skew 500 ppm
I20260928 14:51:31.417564 24625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:51:31.419891 24625 webserver.cc:460] Webserver started at http://127.14.241.254:34689/ using document root <none> and password file <none>
I20260928 14:51:31.420607 24625 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:31.420710 24625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:31.421026 24625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:31.423879 24625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "eaabac1b089240acacd9df101ccf359b"
format_stamp: "Formatted at 2026-09-28 14:51:31 on dist-test-slave-vk40"
server_key: "c9520ed54a5c26e95a53b9355bce6932"
server_key_iv: "6733db98ffdc7e26004f7e563e376a43"
server_key_version: "encryptionkey@0"
I20260928 14:51:31.424675 24625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "eaabac1b089240acacd9df101ccf359b"
format_stamp: "Formatted at 2026-09-28 14:51:31 on dist-test-slave-vk40"
server_key: "c9520ed54a5c26e95a53b9355bce6932"
server_key_iv: "6733db98ffdc7e26004f7e563e376a43"
server_key_version: "encryptionkey@0"
I20260928 14:51:31.429773 24625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260928 14:51:31.433357 24639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:31.434719 24625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 14:51:31.434883 24625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "eaabac1b089240acacd9df101ccf359b"
format_stamp: "Formatted at 2026-09-28 14:51:31 on dist-test-slave-vk40"
server_key: "c9520ed54a5c26e95a53b9355bce6932"
server_key_iv: "6733db98ffdc7e26004f7e563e376a43"
server_key_version: "encryptionkey@0"
I20260928 14:51:31.435081 24625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:31.453639 24625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:31.454481 24625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:31.454742 24625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:31.471118 24625 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:33321
I20260928 14:51:31.471138 24690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:33321 every 8 connection(s)
I20260928 14:51:31.472451 24625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:51:31.475996 24691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:31.480667 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 24625
I20260928 14:51:31.480835 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:51:31.481452 15303 external_mini_cluster.cc:1428] Setting key e37824ff60760cc37079931f71e44318
I20260928 14:51:31.487565 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b: Bootstrap starting.
I20260928 14:51:31.491389 24691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:31.492660 24691 log.cc:824] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:51:31.496166 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b: No bootstrap required, opened a new log
I20260928 14:51:31.497131 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:31.483430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43576 (local address 127.14.241.254:33321)
0928 14:51:31.484599 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:31.484614 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:31.484669 (+ 55us) server_negotiation.cc:407] Connection header received
0928 14:51:31.486933 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:31.486960 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:31.487580 (+ 620us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:31.488011 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:31.488578 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:31.490527 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:31.491899 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:31.492977 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:31.493167 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:31.494277 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:31.494287 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:31.494306 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:31.495014 (+ 708us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:31.495114 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:31.495142 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:31.495265 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:31.495390 (+ 125us) server_negotiation.cc:299] Negotiation successful
0928 14:51:31.495771 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":332,"threads_started":1}
I20260928 14:51:31.504570 24691 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaabac1b089240acacd9df101ccf359b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33321 } }
I20260928 14:51:31.505087 24691 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:31.505199 24691 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaabac1b089240acacd9df101ccf359b, State: Initialized, Role: FOLLOWER
I20260928 14:51:31.505815 24691 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaabac1b089240acacd9df101ccf359b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33321 } }
I20260928 14:51:31.506055 24691 raft_consensus.cc:388] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:51:31.506181 24691 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:51:31.506340 24691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:31.508582 24691 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaabac1b089240acacd9df101ccf359b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33321 } }
I20260928 14:51:31.509105 24691 leader_election.cc:302] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eaabac1b089240acacd9df101ccf359b; no voters:
I20260928 14:51:31.509662 24691 leader_election.cc:288] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:51:31.509848 24696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:31.510464 24696 raft_consensus.cc:686] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [term 1 LEADER]: Becoming Leader. State: Replica: eaabac1b089240acacd9df101ccf359b, State: Running, Role: LEADER
I20260928 14:51:31.510957 24696 consensus_queue.cc:237] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaabac1b089240acacd9df101ccf359b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33321 } }
I20260928 14:51:31.511861 24691 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:51:31.513743 24697 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eaabac1b089240acacd9df101ccf359b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaabac1b089240acacd9df101ccf359b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33321 } } }
I20260928 14:51:31.514204 24697 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:31.513744 24698 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaabac1b089240acacd9df101ccf359b. Latest consensus state: current_term: 1 leader_uuid: "eaabac1b089240acacd9df101ccf359b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaabac1b089240acacd9df101ccf359b" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33321 } } }
I20260928 14:51:31.514706 24698 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:31.516829 24703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:51:31.521320 24703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:51:31.529958 24703 catalog_manager.cc:1347] Generated new cluster ID: b7259467f2764c79a78d84dc0be12a09
I20260928 14:51:31.530076 24703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:51:31.566576 24703 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:51:31.567914 24703 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:51:31.575218 24703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eaabac1b089240acacd9df101ccf359b: Generated new TSK 0
I20260928 14:51:31.576081 24703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:51:31.585086 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33321
--builtin_ntp_servers=127.14.241.212:44223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260928 14:51:31.903990 24715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:31.904459 24715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:31.904578 24715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:31.911917 24715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:31.912048 24715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:31.912202 24715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:51:31.921487 24715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:31.922760 24715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:31.924523 24715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:31.936851 24720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:31.936863 24721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:31.937034 24723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:31.937654 24715 server_base.cc:1034] running on GCE node
I20260928 14:51:31.938251 24715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:31.939204 24715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:31.940363 24715 hybrid_clock.cc:648] HybridClock initialized: now 1790607091940307 us; error 49 us; skew 500 ppm
I20260928 14:51:31.940653 24715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:51:31.948642 24715 webserver.cc:460] Webserver started at http://127.14.241.193:35605/ using document root <none> and password file <none>
I20260928 14:51:31.949343 24715 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:31.949434 24715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:31.949702 24715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:31.952530 24715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "706067260478411b804203857500408e"
format_stamp: "Formatted at 2026-09-28 14:51:31 on dist-test-slave-vk40"
server_key: "71abebba44e4ff77418d567244703b19"
server_key_iv: "45789ed024256edb196371c7b99116c0"
server_key_version: "encryptionkey@0"
I20260928 14:51:31.953255 24715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "706067260478411b804203857500408e"
format_stamp: "Formatted at 2026-09-28 14:51:31 on dist-test-slave-vk40"
server_key: "71abebba44e4ff77418d567244703b19"
server_key_iv: "45789ed024256edb196371c7b99116c0"
server_key_version: "encryptionkey@0"
I20260928 14:51:31.958268 24715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:51:31.961493 24729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:31.962808 24715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:51:31.962970 24715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "706067260478411b804203857500408e"
format_stamp: "Formatted at 2026-09-28 14:51:31 on dist-test-slave-vk40"
server_key: "71abebba44e4ff77418d567244703b19"
server_key_iv: "45789ed024256edb196371c7b99116c0"
server_key_version: "encryptionkey@0"
I20260928 14:51:31.963166 24715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:31.989702 24715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:31.990578 24715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:31.990854 24715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:31.991813 24715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:31.993376 24715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:31.993484 24715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:31.993592 24715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:31.993660 24715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:32.023289 24715 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:41165
I20260928 14:51:32.023360 24841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:41165 every 8 connection(s)
I20260928 14:51:32.024623 24715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:51:32.026503 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 24715
I20260928 14:51:32.026701 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:51:32.027482 15303 external_mini_cluster.cc:1428] Setting key 5b81c1906eced55d6ba77c586e5a1133
I20260928 14:51:32.030336 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33321
--builtin_ntp_servers=127.14.241.212:44223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 14:51:32.039218 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:32.027150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:46779 (local address 127.14.241.254:33321)
0928 14:51:32.027577 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:32.027583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:32.030564 (+ 2981us) server_negotiation.cc:407] Connection header received
0928 14:51:32.031487 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:32.031490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:32.031554 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:32.031641 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:32.033104 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.034063 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.036194 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.036720 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.036822 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:32.038327 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:32.038332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:32.038336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:32.038354 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:32.038398 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:32.038422 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:32.038529 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:32.038785 (+ 256us) server_negotiation.cc:299] Negotiation successful
0928 14:51:32.038867 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":221,"threads_started":1}
I20260928 14:51:32.040386 24845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:32.027908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33321 (local address 127.14.241.193:46779)
0928 14:51:32.028998 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:32.029043 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:32.031174 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:32.031914 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:32.031926 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:32.032055 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:32.032891 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:32.032928 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.034216 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.034222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:32.036049 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:32.036065 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.036947 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.036953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:32.037234 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:32.037986 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:51:32.037989 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:32.038036 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:51:32.038142 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:32.038551 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:32.038564 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:32.038573 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 14:51:32.038787 (+ 214us) client_negotiation.cc:240] Negotiation successful
0928 14:51:32.039184 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":326,"threads_started":1}
I20260928 14:51:32.041932 24842 heartbeater.cc:347] Connected to a master server at 127.14.241.254:33321
I20260928 14:51:32.042258 24842 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:32.043126 24842 heartbeater.cc:511] Master 127.14.241.254:33321 requested a full tablet report, sending...
I20260928 14:51:32.045269 24656 ts_manager.cc:194] Registered new tserver with Master: 706067260478411b804203857500408e (127.14.241.193:41165)
I20260928 14:51:32.046777 24656 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.193:46779
W20260928 14:51:32.350387 24846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:32.350682 24846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:32.350809 24846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:32.358134 24846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:32.358250 24846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:32.358399 24846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:51:32.367607 24846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:32.368933 24846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:32.370673 24846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:32.383723 24851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:32.383787 24852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:32.383811 24854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:32.384825 24846 server_base.cc:1034] running on GCE node
I20260928 14:51:32.385406 24846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:32.386309 24846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:32.387497 24846 hybrid_clock.cc:648] HybridClock initialized: now 1790607092387445 us; error 45 us; skew 500 ppm
I20260928 14:51:32.387799 24846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:51:32.389997 24846 webserver.cc:460] Webserver started at http://127.14.241.194:46467/ using document root <none> and password file <none>
I20260928 14:51:32.390653 24846 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:32.390751 24846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:32.391059 24846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:32.393945 24846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "17bd4fc865cc4947b8255e806d66ec06"
format_stamp: "Formatted at 2026-09-28 14:51:32 on dist-test-slave-vk40"
server_key: "741df14c02a1262a1b326c681f1fc5e5"
server_key_iv: "ed06e49ffefff97be1fa9dd92452adba"
server_key_version: "encryptionkey@0"
I20260928 14:51:32.394719 24846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "17bd4fc865cc4947b8255e806d66ec06"
format_stamp: "Formatted at 2026-09-28 14:51:32 on dist-test-slave-vk40"
server_key: "741df14c02a1262a1b326c681f1fc5e5"
server_key_iv: "ed06e49ffefff97be1fa9dd92452adba"
server_key_version: "encryptionkey@0"
I20260928 14:51:32.400089 24846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260928 14:51:32.403486 24860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:32.404872 24846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:51:32.405032 24846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "17bd4fc865cc4947b8255e806d66ec06"
format_stamp: "Formatted at 2026-09-28 14:51:32 on dist-test-slave-vk40"
server_key: "741df14c02a1262a1b326c681f1fc5e5"
server_key_iv: "ed06e49ffefff97be1fa9dd92452adba"
server_key_version: "encryptionkey@0"
I20260928 14:51:32.405258 24846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:32.418212 24846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:32.419034 24846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:32.419282 24846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:32.420174 24846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:32.421693 24846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:32.421782 24846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:32.421871 24846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:32.421926 24846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:32.451712 24846 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:46763
I20260928 14:51:32.451758 24972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:46763 every 8 connection(s)
I20260928 14:51:32.453066 24846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:51:32.454376 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 24846
I20260928 14:51:32.454519 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:51:32.454989 15303 external_mini_cluster.cc:1428] Setting key 5e37db66288b0c00311846423535efcf
I20260928 14:51:32.457502 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33321
--builtin_ntp_servers=127.14.241.212:44223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 14:51:32.465533 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:32.455581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:35977 (local address 127.14.241.254:33321)
0928 14:51:32.455789 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:32.455795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:32.457380 (+ 1585us) server_negotiation.cc:407] Connection header received
0928 14:51:32.459068 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:32.459074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:32.459151 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:32.459251 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:32.460323 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.461247 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.462788 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.463298 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.463418 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:32.464785 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:32.464792 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:32.464796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:32.464816 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:32.464852 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:32.464865 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:32.464941 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:32.465119 (+ 178us) server_negotiation.cc:299] Negotiation successful
0928 14:51:32.465228 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:51:32.466643 24975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:32.456160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33321 (local address 127.14.241.194:35977)
0928 14:51:32.457207 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:32.457258 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:32.458818 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:32.459443 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:32.459451 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:32.459547 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:32.460141 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:32.460158 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.461403 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.461407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:32.462651 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:32.462663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.463525 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.463531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:32.463756 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:32.464474 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:51:32.464477 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:32.464522 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:51:32.464605 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:32.464951 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:32.464958 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:32.464963 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:51:32.465105 (+ 142us) client_negotiation.cc:240] Negotiation successful
0928 14:51:32.465430 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":291,"threads_started":1}
I20260928 14:51:32.468163 24973 heartbeater.cc:347] Connected to a master server at 127.14.241.254:33321
I20260928 14:51:32.468472 24973 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:32.469234 24973 heartbeater.cc:511] Master 127.14.241.254:33321 requested a full tablet report, sending...
I20260928 14:51:32.470695 24656 ts_manager.cc:194] Registered new tserver with Master: 17bd4fc865cc4947b8255e806d66ec06 (127.14.241.194:46763)
I20260928 14:51:32.471717 24656 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.194:35977
W20260928 14:51:32.777613 24976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:32.777904 24976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:32.777995 24976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:32.785122 24976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:32.785221 24976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:32.785329 24976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:51:32.794298 24976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:33321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:32.795554 24976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:32.797310 24976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:32.809855 24982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:32.810038 24984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:32.809819 24981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:32.810485 24976 server_base.cc:1034] running on GCE node
I20260928 14:51:32.811100 24976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:32.812040 24976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:32.813233 24976 hybrid_clock.cc:648] HybridClock initialized: now 1790607092813164 us; error 63 us; skew 500 ppm
I20260928 14:51:32.813548 24976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:51:32.815639 24976 webserver.cc:460] Webserver started at http://127.14.241.195:36633/ using document root <none> and password file <none>
I20260928 14:51:32.816360 24976 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:32.816461 24976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:32.816754 24976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:32.819533 24976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "8b646674aae142c28f273cd90eb84282"
format_stamp: "Formatted at 2026-09-28 14:51:32 on dist-test-slave-vk40"
server_key: "a75a3acb2c15f5918e69525eb8cdf186"
server_key_iv: "26b5692bb3c41afd338d38cdaa346125"
server_key_version: "encryptionkey@0"
I20260928 14:51:32.820348 24976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "8b646674aae142c28f273cd90eb84282"
format_stamp: "Formatted at 2026-09-28 14:51:32 on dist-test-slave-vk40"
server_key: "a75a3acb2c15f5918e69525eb8cdf186"
server_key_iv: "26b5692bb3c41afd338d38cdaa346125"
server_key_version: "encryptionkey@0"
I20260928 14:51:32.825218 24976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260928 14:51:32.828473 24990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:32.829752 24976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 14:51:32.829901 24976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "8b646674aae142c28f273cd90eb84282"
format_stamp: "Formatted at 2026-09-28 14:51:32 on dist-test-slave-vk40"
server_key: "a75a3acb2c15f5918e69525eb8cdf186"
server_key_iv: "26b5692bb3c41afd338d38cdaa346125"
server_key_version: "encryptionkey@0"
I20260928 14:51:32.830116 24976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:32.866474 24976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:32.867332 24976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:32.867609 24976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:32.868508 24976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:32.869884 24976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:32.869969 24976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:32.870046 24976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:32.870096 24976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:32.899029 24976 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:42201
I20260928 14:51:32.899084 25102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:42201 every 8 connection(s)
I20260928 14:51:32.900252 24976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:51:32.901355 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 24976
I20260928 14:51:32.901516 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:51:32.902048 15303 external_mini_cluster.cc:1428] Setting key 8d7010e1063fdfbba443787492e7dbac
I20260928 14:51:32.912367 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:32.902829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:36863 (local address 127.14.241.254:33321)
0928 14:51:32.903038 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:32.903045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:32.904627 (+ 1582us) server_negotiation.cc:407] Connection header received
0928 14:51:32.906500 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:32.906504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:32.906573 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:32.906646 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:32.907645 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.908409 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.909917 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.910354 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.910453 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:32.911675 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:32.911680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:32.911683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:32.911703 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:32.911735 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:32.911746 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:32.911814 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:32.911989 (+ 175us) server_negotiation.cc:299] Negotiation successful
0928 14:51:32.912068 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260928 14:51:32.913380 25105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:32.903426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:33321 (local address 127.14.241.195:36863)
0928 14:51:32.904463 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:32.904519 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:32.906249 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:32.906854 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:32.906862 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:32.906951 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:32.907520 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:32.907537 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.908524 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.908528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:32.909778 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:32.909790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.910548 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.910553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:32.910757 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:32.911375 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:51:32.911378 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:32.911416 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:51:32.911499 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:32.911814 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:32.911820 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:32.911825 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:51:32.911972 (+ 147us) client_negotiation.cc:240] Negotiation successful
0928 14:51:32.912335 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":262,"threads_started":1}
I20260928 14:51:32.914636 25103 heartbeater.cc:347] Connected to a master server at 127.14.241.254:33321
I20260928 14:51:32.914923 25103 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:32.915777 25103 heartbeater.cc:511] Master 127.14.241.254:33321 requested a full tablet report, sending...
I20260928 14:51:32.917383 24655 ts_manager.cc:194] Registered new tserver with Master: 8b646674aae142c28f273cd90eb84282 (127.14.241.195:42201)
I20260928 14:51:32.918220 24655 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.195:36863
I20260928 14:51:32.929602 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 14:51:32.935329 24844 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:32.933980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.14.241.254:33321)
0928 14:51:32.934189 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:32.934194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:32.934557 (+ 363us) server_negotiation.cc:407] Connection header received
0928 14:51:32.934702 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:32.934705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:32.934777 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:32.934857 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:32.935145 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34242: BlockingRecv error: recv got EOF from 127.0.0.1:34242 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:51:32.935928 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 24715
I20260928 14:51:32.943174 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:32.936795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44178 (local address 127.14.241.193:41165)
0928 14:51:32.937250 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:32.937259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:32.937288 (+ 29us) server_negotiation.cc:407] Connection header received
0928 14:51:32.937351 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:32.937357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:32.937587 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:32.937740 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:32.938034 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.939061 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.940332 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:32.940895 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:32.940989 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:32.941869 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:32.941876 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:32.941890 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:32.942481 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:32.942529 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:32.942548 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:32.942629 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:32.942721 (+ 92us) server_negotiation.cc:299] Negotiation successful
0928 14:51:32.942826 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":222,"threads_started":1}
I20260928 14:51:32.944386 24837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:33.024477 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 24715
I20260928 14:51:33.038267 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 24846
I20260928 14:51:33.045555 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:33.039108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.14.241.194:46763)
0928 14:51:33.039657 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:33.039664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:33.039690 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:51:33.039751 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:33.039757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:33.039991 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:33.040121 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:33.040486 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:33.041488 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:33.042628 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:33.043172 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:33.043263 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:33.044114 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:33.044122 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:33.044134 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:33.044775 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:33.044822 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:33.044842 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:33.044932 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:33.045067 (+ 135us) server_negotiation.cc:299] Negotiation successful
0928 14:51:33.045199 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":285,"threads_started":1}
I20260928 14:51:33.046810 24968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:33.128602 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 24846
I20260928 14:51:33.141912 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 24976
I20260928 14:51:33.149497 25116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:33.142778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52278 (local address 127.14.241.195:42201)
0928 14:51:33.143427 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:33.143435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:33.143466 (+ 31us) server_negotiation.cc:407] Connection header received
0928 14:51:33.143528 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:33.143534 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:33.143782 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:33.143914 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:33.144365 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:33.145407 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:33.146566 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:33.147146 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:33.147245 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:33.148123 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:33.148130 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:33.148143 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:33.148755 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:33.148799 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:33.148818 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:33.148914 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:33.149041 (+ 127us) server_negotiation.cc:299] Negotiation successful
0928 14:51:33.149149 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":338,"threads_started":1}
I20260928 14:51:33.150703 25098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:33.231840 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 24976
I20260928 14:51:33.245148 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 24625
I20260928 14:51:33.246279 24686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:33.348318 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 24625
2026-09-28T14:51:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2354 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-28T14:51:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:51:33Z Disabled control of system clock
I20260928 14:51:33.415115 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:45555
--webserver_interface=127.14.241.254
--webserver_port=0
--builtin_ntp_servers=127.14.241.212:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:45555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 14:51:33.736023 25122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:33.736361 25122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:33.736464 25122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:33.743716 25122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:51:33.743829 25122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:33.743865 25122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:33.743927 25122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:51:33.743970 25122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:51:33.753253 25122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:45555
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:45555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:33.754513 25122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:33.756309 25122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:33.767400 25130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:33.767477 25127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:33.767477 25128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:33.768344 25122 server_base.cc:1034] running on GCE node
I20260928 14:51:33.769116 25122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:33.770483 25122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:33.771682 25122 hybrid_clock.cc:648] HybridClock initialized: now 1790607093771641 us; error 33 us; skew 500 ppm
I20260928 14:51:33.771996 25122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:51:33.774088 25122 webserver.cc:460] Webserver started at http://127.14.241.254:41973/ using document root <none> and password file <none>
I20260928 14:51:33.774715 25122 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:33.774801 25122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:33.775070 25122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:33.777873 25122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "99bc6e9cf6ee4ab09a825730fb01c84a"
format_stamp: "Formatted at 2026-09-28 14:51:33 on dist-test-slave-vk40"
server_key: "d392282a679d6f31b0482ddc46d86ea7"
server_key_iv: "a855ed3cc2874bee39af11a6f89b7c9e"
server_key_version: "encryptionkey@0"
I20260928 14:51:33.778589 25122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "99bc6e9cf6ee4ab09a825730fb01c84a"
format_stamp: "Formatted at 2026-09-28 14:51:33 on dist-test-slave-vk40"
server_key: "d392282a679d6f31b0482ddc46d86ea7"
server_key_iv: "a855ed3cc2874bee39af11a6f89b7c9e"
server_key_version: "encryptionkey@0"
I20260928 14:51:33.783440 25122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:51:33.786700 25136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:33.788004 25122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:51:33.788133 25122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "99bc6e9cf6ee4ab09a825730fb01c84a"
format_stamp: "Formatted at 2026-09-28 14:51:33 on dist-test-slave-vk40"
server_key: "d392282a679d6f31b0482ddc46d86ea7"
server_key_iv: "a855ed3cc2874bee39af11a6f89b7c9e"
server_key_version: "encryptionkey@0"
I20260928 14:51:33.788333 25122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:33.793833 25122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:33.794546 25122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:33.794791 25122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:33.811299 25122 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:45555
I20260928 14:51:33.811329 25187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:45555 every 8 connection(s)
I20260928 14:51:33.812602 25122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:51:33.815651 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 25122
I20260928 14:51:33.815855 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:51:33.816520 25188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:33.816654 15303 external_mini_cluster.cc:1428] Setting key f9b802004db7451b9a6207f66cf2448d
I20260928 14:51:33.824452 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:33.818632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.14.241.254:45555)
0928 14:51:33.819602 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:33.819613 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:33.819657 (+ 44us) server_negotiation.cc:407] Connection header received
0928 14:51:33.821139 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:33.821160 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:33.821575 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:33.821900 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:33.822913 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:33.822923 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:33.822935 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:33.822956 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:33.823013 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:33.823028 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:33.823187 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:33.823277 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 14:51:33.823536 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":344,"threads_started":1}
I20260928 14:51:33.829006 25188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a: Bootstrap starting.
I20260928 14:51:33.831754 25188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:33.832762 25188 log.cc:824] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:51:33.835170 25188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a: No bootstrap required, opened a new log
I20260928 14:51:33.842191 25188 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99bc6e9cf6ee4ab09a825730fb01c84a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45555 } }
I20260928 14:51:33.842523 25188 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:33.842592 25188 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99bc6e9cf6ee4ab09a825730fb01c84a, State: Initialized, Role: FOLLOWER
I20260928 14:51:33.843114 25188 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99bc6e9cf6ee4ab09a825730fb01c84a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45555 } }
I20260928 14:51:33.843312 25188 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:51:33.843420 25188 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:51:33.843529 25188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:33.845814 25188 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99bc6e9cf6ee4ab09a825730fb01c84a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45555 } }
I20260928 14:51:33.846244 25188 leader_election.cc:302] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99bc6e9cf6ee4ab09a825730fb01c84a; no voters:
I20260928 14:51:33.846812 25188 leader_election.cc:288] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:51:33.846918 25193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:33.847571 25193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [term 1 LEADER]: Becoming Leader. State: Replica: 99bc6e9cf6ee4ab09a825730fb01c84a, State: Running, Role: LEADER
I20260928 14:51:33.848109 25193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99bc6e9cf6ee4ab09a825730fb01c84a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45555 } }
I20260928 14:51:33.848738 25188 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:51:33.850680 25195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99bc6e9cf6ee4ab09a825730fb01c84a. Latest consensus state: current_term: 1 leader_uuid: "99bc6e9cf6ee4ab09a825730fb01c84a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99bc6e9cf6ee4ab09a825730fb01c84a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45555 } } }
I20260928 14:51:33.851049 25195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:33.852248 25194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99bc6e9cf6ee4ab09a825730fb01c84a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99bc6e9cf6ee4ab09a825730fb01c84a" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 45555 } } }
I20260928 14:51:33.852631 25194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:33.853889 25201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:51:33.858561 25201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:51:33.867030 25201 catalog_manager.cc:1347] Generated new cluster ID: ab34f3f29baa4636931b1061474dfe9f
I20260928 14:51:33.867146 25201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:51:33.891688 25201 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:51:33.892316 25201 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:51:33.899240 25201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99bc6e9cf6ee4ab09a825730fb01c84a: Generated new TSK 0
I20260928 14:51:33.900264 25201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:51:33.908558 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:45555
--builtin_ntp_servers=127.14.241.212:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 14:51:34.224702 25212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:34.224997 25212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:34.225131 25212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:34.233462 25212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:34.233578 25212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:34.233733 25212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:51:34.242987 25212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:45555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:34.244302 25212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:34.246027 25212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:34.258731 25218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:34.258805 25217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:34.258786 25220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:34.259593 25212 server_base.cc:1034] running on GCE node
I20260928 14:51:34.260183 25212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:34.261184 25212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:34.262379 25212 hybrid_clock.cc:648] HybridClock initialized: now 1790607094262330 us; error 44 us; skew 500 ppm
I20260928 14:51:34.262696 25212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:51:34.264977 25212 webserver.cc:460] Webserver started at http://127.14.241.193:44399/ using document root <none> and password file <none>
I20260928 14:51:34.265674 25212 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:34.265774 25212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:34.266069 25212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:34.268934 25212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "cbbe080861954896abc74628d52b8854"
format_stamp: "Formatted at 2026-09-28 14:51:34 on dist-test-slave-vk40"
server_key: "9d20b769498bd3e097b7234811cb079e"
server_key_iv: "089117b8d109b8086974e31d1f12a3c9"
server_key_version: "encryptionkey@0"
I20260928 14:51:34.269697 25212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "cbbe080861954896abc74628d52b8854"
format_stamp: "Formatted at 2026-09-28 14:51:34 on dist-test-slave-vk40"
server_key: "9d20b769498bd3e097b7234811cb079e"
server_key_iv: "089117b8d109b8086974e31d1f12a3c9"
server_key_version: "encryptionkey@0"
I20260928 14:51:34.274693 25212 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:51:34.278112 25226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:34.279572 25212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:51:34.279726 25212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "cbbe080861954896abc74628d52b8854"
format_stamp: "Formatted at 2026-09-28 14:51:34 on dist-test-slave-vk40"
server_key: "9d20b769498bd3e097b7234811cb079e"
server_key_iv: "089117b8d109b8086974e31d1f12a3c9"
server_key_version: "encryptionkey@0"
I20260928 14:51:34.279933 25212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:34.285390 25212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:34.286207 25212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:34.286461 25212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:34.287359 25212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:34.288736 25212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:34.288829 25212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:34.288934 25212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:34.289011 25212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:34.318140 25212 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:34005
I20260928 14:51:34.318195 25338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:34005 every 8 connection(s)
I20260928 14:51:34.319427 25212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:51:34.327394 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:34.321960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:52357 (local address 127.14.241.254:45555)
0928 14:51:34.322138 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:34.322143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:34.323574 (+ 1431us) server_negotiation.cc:407] Connection header received
0928 14:51:34.325300 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:34.325306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:34.325370 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:34.325463 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:34.326697 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:34.326703 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:34.326706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:34.326723 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:34.326755 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:34.326768 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:34.326870 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:34.327062 (+ 192us) server_negotiation.cc:299] Negotiation successful
0928 14:51:34.327129 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:51:34.328311 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:34.322482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:45555 (local address 127.14.241.193:52357)
0928 14:51:34.323417 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:34.323462 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:34.325034 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:34.325696 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:34.325704 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:34.325797 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:34.326398 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:51:34.326401 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:34.326440 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:51:34.326523 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:34.326851 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:34.326861 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:34.326868 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 14:51:34.327130 (+ 262us) client_negotiation.cc:240] Negotiation successful
0928 14:51:34.327383 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":253,"threads_started":1}
I20260928 14:51:34.329416 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 25212
I20260928 14:51:34.329595 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:51:34.329695 25339 heartbeater.cc:347] Connected to a master server at 127.14.241.254:45555
I20260928 14:51:34.330027 25339 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:34.330381 15303 external_mini_cluster.cc:1428] Setting key b70a9d4363a1f9cabd9d09623be12db4
I20260928 14:51:34.330717 25339 heartbeater.cc:511] Master 127.14.241.254:45555 requested a full tablet report, sending...
I20260928 14:51:34.332896 25152 ts_manager.cc:194] Registered new tserver with Master: cbbe080861954896abc74628d52b8854 (127.14.241.193:34005)
I20260928 14:51:34.333088 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:45555
--builtin_ntp_servers=127.14.241.212:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 14:51:34.648797 25342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:34.649109 25342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:34.649209 25342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:34.656666 25342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:34.656770 25342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:34.656924 25342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:51:34.665967 25342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:45555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:34.667239 25342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:34.668956 25342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:34.681205 25347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:34.681160 25350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:34.681236 25348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:34.682019 25342 server_base.cc:1034] running on GCE node
I20260928 14:51:34.682626 25342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:34.683504 25342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:34.684690 25342 hybrid_clock.cc:648] HybridClock initialized: now 1790607094684649 us; error 34 us; skew 500 ppm
I20260928 14:51:34.684974 25342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:51:34.687016 25342 webserver.cc:460] Webserver started at http://127.14.241.194:44941/ using document root <none> and password file <none>
I20260928 14:51:34.687673 25342 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:34.687763 25342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:34.688028 25342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:34.690939 25342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "b28c0de539724e719de85b28e3f3a0e6"
format_stamp: "Formatted at 2026-09-28 14:51:34 on dist-test-slave-vk40"
server_key: "d493faed361f46d213babc042a21b3a3"
server_key_iv: "7dd8162473df53e2bc8ac8d42477fc94"
server_key_version: "encryptionkey@0"
I20260928 14:51:34.691709 25342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "b28c0de539724e719de85b28e3f3a0e6"
format_stamp: "Formatted at 2026-09-28 14:51:34 on dist-test-slave-vk40"
server_key: "d493faed361f46d213babc042a21b3a3"
server_key_iv: "7dd8162473df53e2bc8ac8d42477fc94"
server_key_version: "encryptionkey@0"
I20260928 14:51:34.696821 25342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260928 14:51:34.700203 25356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:34.701573 25342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:51:34.701706 25342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "b28c0de539724e719de85b28e3f3a0e6"
format_stamp: "Formatted at 2026-09-28 14:51:34 on dist-test-slave-vk40"
server_key: "d493faed361f46d213babc042a21b3a3"
server_key_iv: "7dd8162473df53e2bc8ac8d42477fc94"
server_key_version: "encryptionkey@0"
I20260928 14:51:34.701884 25342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:34.707427 25342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:34.708156 25342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:34.708421 25342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:34.709234 25342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:34.710765 25342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:34.710850 25342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:34.710961 25342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:34.711025 25342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:34.740092 25342 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:46515
I20260928 14:51:34.740142 25468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:46515 every 8 connection(s)
I20260928 14:51:34.741379 25342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:51:34.743638 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 25342
I20260928 14:51:34.743795 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:51:34.744261 15303 external_mini_cluster.cc:1428] Setting key feb9d0c71c356cf83990962e000b9989
I20260928 14:51:34.747350 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:45555
--builtin_ntp_servers=127.14.241.212:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260928 14:51:34.749832 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:34.743824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:54665 (local address 127.14.241.254:45555)
0928 14:51:34.744033 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:34.744040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:34.745524 (+ 1484us) server_negotiation.cc:407] Connection header received
0928 14:51:34.747667 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:34.747672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:34.747722 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:34.747806 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:34.749228 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:34.749232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:34.749235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:34.749251 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:34.749284 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:34.749296 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:34.749382 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:34.749573 (+ 191us) server_negotiation.cc:299] Negotiation successful
0928 14:51:34.749624 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 14:51:34.750993 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:34.744379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:45555 (local address 127.14.241.194:54665)
0928 14:51:34.745381 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:34.745424 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:34.747322 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:34.748055 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:34.748067 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:34.748193 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:34.748901 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:51:34.748904 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:34.748951 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:51:34.749056 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:34.749381 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:34.749391 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:34.749398 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 14:51:34.749656 (+ 258us) client_negotiation.cc:240] Negotiation successful
0928 14:51:34.749912 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":284,"threads_started":1}
I20260928 14:51:34.752254 25469 heartbeater.cc:347] Connected to a master server at 127.14.241.254:45555
I20260928 14:51:34.752552 25469 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:34.753141 25469 heartbeater.cc:511] Master 127.14.241.254:45555 requested a full tablet report, sending...
I20260928 14:51:34.754478 25152 ts_manager.cc:194] Registered new tserver with Master: b28c0de539724e719de85b28e3f3a0e6 (127.14.241.194:46515)
W20260928 14:51:35.064352 25472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:35.064620 25472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:35.064710 25472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:35.071791 25472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:35.071885 25472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:35.072009 25472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:51:35.080999 25472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:45555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:35.082218 25472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:35.083966 25472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:35.096334 25477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:35.096441 25480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:35.096472 25478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:35.097550 25472 server_base.cc:1034] running on GCE node
I20260928 14:51:35.098152 25472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:35.099115 25472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:35.100317 25472 hybrid_clock.cc:648] HybridClock initialized: now 1790607095100239 us; error 50 us; skew 500 ppm
I20260928 14:51:35.100617 25472 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:51:35.102752 25472 webserver.cc:460] Webserver started at http://127.14.241.195:38471/ using document root <none> and password file <none>
I20260928 14:51:35.103425 25472 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:35.103518 25472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:35.103825 25472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:35.106815 25472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "ecfd40274ba943d68ada92a6a601fcbe"
format_stamp: "Formatted at 2026-09-28 14:51:35 on dist-test-slave-vk40"
server_key: "c89740981e874f1da880f62c1baf8799"
server_key_iv: "55688a6d194a3041d3e86b6226f30088"
server_key_version: "encryptionkey@0"
I20260928 14:51:35.107605 25472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "ecfd40274ba943d68ada92a6a601fcbe"
format_stamp: "Formatted at 2026-09-28 14:51:35 on dist-test-slave-vk40"
server_key: "c89740981e874f1da880f62c1baf8799"
server_key_iv: "55688a6d194a3041d3e86b6226f30088"
server_key_version: "encryptionkey@0"
I20260928 14:51:35.118866 25472 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.010s sys 0.003s
I20260928 14:51:35.123268 25486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:35.125097 25472 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260928 14:51:35.125270 25472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "ecfd40274ba943d68ada92a6a601fcbe"
format_stamp: "Formatted at 2026-09-28 14:51:35 on dist-test-slave-vk40"
server_key: "c89740981e874f1da880f62c1baf8799"
server_key_iv: "55688a6d194a3041d3e86b6226f30088"
server_key_version: "encryptionkey@0"
I20260928 14:51:35.125528 25472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:35.131258 25472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:35.132056 25472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:35.132332 25472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:35.133219 25472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:35.134603 25472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:35.134696 25472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:35.134800 25472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:35.134871 25472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:35.163460 25472 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:46691
I20260928 14:51:35.163504 25598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:46691 every 8 connection(s)
I20260928 14:51:35.164783 25472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:51:35.169636 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 25472
I20260928 14:51:35.169793 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:51:35.170183 15303 external_mini_cluster.cc:1428] Setting key e2bd6ab234ad653782aadc063185adb3
I20260928 14:51:35.172766 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:35.167215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:43543 (local address 127.14.241.254:45555)
0928 14:51:35.167411 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:35.167416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:35.168802 (+ 1386us) server_negotiation.cc:407] Connection header received
0928 14:51:35.170536 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:35.170543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:35.170612 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:35.170702 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:35.172032 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:35.172039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:35.172043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:35.172061 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:35.172093 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:35.172104 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:35.172208 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:35.172410 (+ 202us) server_negotiation.cc:299] Negotiation successful
0928 14:51:35.172479 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:51:35.173822 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:35.167736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:45555 (local address 127.14.241.195:43543)
0928 14:51:35.168655 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:35.168692 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:35.170258 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:35.170954 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:35.170962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:35.171074 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:35.171737 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:51:35.171739 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:35.171777 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:51:35.171865 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:35.172183 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:35.172194 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:35.172202 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 14:51:35.172513 (+ 311us) client_negotiation.cc:240] Negotiation successful
0928 14:51:35.172793 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":253,"threads_started":1}
I20260928 14:51:35.175010 25599 heartbeater.cc:347] Connected to a master server at 127.14.241.254:45555
I20260928 14:51:35.175297 25599 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:35.175863 25599 heartbeater.cc:511] Master 127.14.241.254:45555 requested a full tablet report, sending...
I20260928 14:51:35.177214 25152 ts_manager.cc:194] Registered new tserver with Master: ecfd40274ba943d68ada92a6a601fcbe (127.14.241.195:46691)
I20260928 14:51:35.185303 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 14:51:35.191015 25191 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:51:35.189637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.14.241.254:45555)
0928 14:51:35.189801 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:35.189805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:35.190292 (+ 487us) server_negotiation.cc:407] Connection header received
0928 14:51:35.190429 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:35.190432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:35.190480 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:35.190557 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:35.190808 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52676: BlockingRecv error: recv got EOF from 127.0.0.1:52676 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:51:35.191524 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 25212
I20260928 14:51:35.195022 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:35.192624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54256 (local address 127.14.241.193:34005)
0928 14:51:35.193154 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:35.193160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:35.193185 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:51:35.193239 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:35.193245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:35.193464 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:35.193585 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:35.194415 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:35.194422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:35.194434 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:35.194451 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:35.194490 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:35.194504 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:35.194604 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:35.194654 (+ 50us) server_negotiation.cc:299] Negotiation successful
0928 14:51:35.194739 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":276,"threads_started":1}
I20260928 14:51:35.196174 25334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:35.278321 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 25212
I20260928 14:51:35.292462 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 25342
I20260928 14:51:35.295969 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:35.293548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35236 (local address 127.14.241.194:46515)
0928 14:51:35.294042 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:35.294049 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:35.294074 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:51:35.294137 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:35.294143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:35.294371 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:35.294498 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:35.295341 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:35.295346 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:35.295368 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:35.295385 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:35.295424 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:35.295437 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:35.295538 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:35.295593 (+ 55us) server_negotiation.cc:299] Negotiation successful
0928 14:51:35.295675 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":235,"threads_started":1}
I20260928 14:51:35.297348 25464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:35.377754 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 25342
I20260928 14:51:35.390798 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 25472
I20260928 14:51:35.394586 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:35.391803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53030 (local address 127.14.241.195:46691)
0928 14:51:35.392484 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:35.392491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:35.392521 (+ 30us) server_negotiation.cc:407] Connection header received
0928 14:51:35.392581 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:35.392587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:35.392809 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:35.392965 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:35.393932 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:35.393938 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:35.393950 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:51:35.393966 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:35.394004 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:51:35.394017 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:35.394141 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:35.394197 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 14:51:35.394286 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":364,"threads_started":1}
I20260928 14:51:35.395758 25594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:35.475381 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 25472
I20260928 14:51:35.488653 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 25122
I20260928 14:51:35.489799 25183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:35.591122 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 25122
2026-09-28T14:51:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2240 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:51:35 dist-test-slave-vk40 krb5kdc[25617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:51:35 dist-test-slave-vk40 krb5kdc[25617](info): set up 2 sockets
Sep 28 14:51:35 dist-test-slave-vk40 krb5kdc[25617](info): commencing operation
krb5kdc: starting...
W20260928 14:51:36.611711 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:51:36 dist-test-slave-vk40 krb5kdc[25617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:51:36.832199 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46089
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:46089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:37.156926 25630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:37.157203 25630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:37.157296 25630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:37.164867 25630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:51:37.164987 25630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:37.165040 25630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:37.165098 25630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:51:37.165146 25630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:51:37.175114 25630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:46089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:37.176452 25630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:37.177708 25630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:37.189292 25635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:37.189257 25638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:37.189366 25636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:37.189972 25630 server_base.cc:1034] running on GCE node
I20260928 14:51:37.190707 25630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:37.192163 25630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:37.193418 25630 hybrid_clock.cc:648] HybridClock initialized: now 1790607097193363 us; error 67 us; skew 500 ppm
I20260928 14:51:37.193792 25630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:37 dist-test-slave-vk40 krb5kdc[25617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607097, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:37.201455 25630 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:51:37.203205 25630 webserver.cc:460] Webserver started at http://127.14.241.254:40081/ using document root <none> and password file <none>
I20260928 14:51:37.203931 25630 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:37.204054 25630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:37.204412 25630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:37.207506 25630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "2b71ff603f824876a418b14ebf3a531f"
format_stamp: "Formatted at 2026-09-28 14:51:37 on dist-test-slave-vk40"
server_key: "ef0373d287876d5cd14b59f311d74a9a"
server_key_iv: "8e7fdc4250717107cbe366f5463921dc"
server_key_version: "encryptionkey@0"
I20260928 14:51:37.208328 25630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "2b71ff603f824876a418b14ebf3a531f"
format_stamp: "Formatted at 2026-09-28 14:51:37 on dist-test-slave-vk40"
server_key: "ef0373d287876d5cd14b59f311d74a9a"
server_key_iv: "8e7fdc4250717107cbe366f5463921dc"
server_key_version: "encryptionkey@0"
I20260928 14:51:37.213676 25630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260928 14:51:37.217485 25645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:37.219000 25630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:51:37.219182 25630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "2b71ff603f824876a418b14ebf3a531f"
format_stamp: "Formatted at 2026-09-28 14:51:37 on dist-test-slave-vk40"
server_key: "ef0373d287876d5cd14b59f311d74a9a"
server_key_iv: "8e7fdc4250717107cbe366f5463921dc"
server_key_version: "encryptionkey@0"
I20260928 14:51:37.219372 25630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:37.248575 25630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:37.252820 25630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:37.253118 25630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:37.270171 25630 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:46089
I20260928 14:51:37.270208 25696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:46089 every 8 connection(s)
I20260928 14:51:37.271517 25630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:51:37.273696 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 25630
I20260928 14:51:37.273907 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:51:37.274506 15303 external_mini_cluster.cc:1428] Setting key c52959f8adad4776fb6173d93bfd60b0
I20260928 14:51:37.275313 25697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 14:51:37 dist-test-slave-vk40 krb5kdc[25617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:51:37.289897 25697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f: Bootstrap starting.
I20260928 14:51:37.293138 25697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:37.294409 25697 log.cc:824] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:51:37.296999 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:37.276388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.14.241.254:46089)
0928 14:51:37.277104 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:37.277116 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:37.277156 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:51:37.278673 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:37.278701 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:37.279101 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:37.279447 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:37.281289 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:37.282633 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:37.284391 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:37.285074 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:37.285206 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:37.289973 (+ 4767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:37.289982 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:37.289996 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:37.290042 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:37.292964 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:37.293777 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:37.293784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:37.293790 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:37.293966 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:37.294340 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:37.294343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:37.294345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:37.294937 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:37.295262 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:37.295595 (+ 333us) server_negotiation.cc:299] Negotiation successful
0928 14:51:37.295970 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":305,"threads_started":1}
I20260928 14:51:37.297477 25697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f: No bootstrap required, opened a new log
I20260928 14:51:37.305377 25697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b71ff603f824876a418b14ebf3a531f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46089 } }
I20260928 14:51:37.305784 25697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:37.305879 25697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b71ff603f824876a418b14ebf3a531f, State: Initialized, Role: FOLLOWER
I20260928 14:51:37.306461 25697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b71ff603f824876a418b14ebf3a531f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46089 } }
I20260928 14:51:37.306706 25697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:51:37.306833 25697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:51:37.307000 25697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:37.309520 25697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b71ff603f824876a418b14ebf3a531f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46089 } }
I20260928 14:51:37.310045 25697 leader_election.cc:302] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b71ff603f824876a418b14ebf3a531f; no voters:
I20260928 14:51:37.310602 25697 leader_election.cc:288] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:51:37.310720 25702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:37.311311 25702 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [term 1 LEADER]: Becoming Leader. State: Replica: 2b71ff603f824876a418b14ebf3a531f, State: Running, Role: LEADER
I20260928 14:51:37.311811 25702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b71ff603f824876a418b14ebf3a531f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46089 } }
I20260928 14:51:37.312559 25697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:51:37.315542 25704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b71ff603f824876a418b14ebf3a531f. Latest consensus state: current_term: 1 leader_uuid: "2b71ff603f824876a418b14ebf3a531f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b71ff603f824876a418b14ebf3a531f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46089 } } }
I20260928 14:51:37.316257 25704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:37.316146 25703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b71ff603f824876a418b14ebf3a531f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b71ff603f824876a418b14ebf3a531f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46089 } } }
I20260928 14:51:37.316522 25703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f [sys.catalog]: This master's current role is: LEADER
I20260928 14:51:37.321090 25711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:51:37.325760 25711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:51:37.334329 25711 catalog_manager.cc:1347] Generated new cluster ID: 375f564a6ccf41d6846e4d8ff862c5c6
I20260928 14:51:37.334439 25711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:51:37.346194 25711 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:51:37.347793 25711 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:51:37.357121 25711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f: Generated new TSK 0
I20260928 14:51:37.357970 25711 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:51:37.407781 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46089
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:37.727567 25723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:37.727843 25723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:37.727972 25723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:37.735147 25723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:37.735265 25723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:37.735412 25723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:51:37.745072 25723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:37.746325 25723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:37.747623 25723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:37.760552 25728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:37.760727 25731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:37.760596 25729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:37.761530 25723 server_base.cc:1034] running on GCE node
I20260928 14:51:37.762195 25723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:37.763146 25723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:37.764376 25723 hybrid_clock.cc:648] HybridClock initialized: now 1790607097764306 us; error 61 us; skew 500 ppm
I20260928 14:51:37.764719 25723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:37 dist-test-slave-vk40 krb5kdc[25617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607097, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:37.768167 25723 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:51:37.769699 25723 webserver.cc:460] Webserver started at http://127.14.241.193:34347/ using document root <none> and password file <none>
I20260928 14:51:37.770395 25723 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:37.770500 25723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:37.770763 25723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:37.773703 25723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d"
format_stamp: "Formatted at 2026-09-28 14:51:37 on dist-test-slave-vk40"
server_key: "9e64ca449b57fd2b734843af0fd752c4"
server_key_iv: "bdbd3ba12bc40543a36e661db82d7aba"
server_key_version: "encryptionkey@0"
I20260928 14:51:37.774434 25723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d"
format_stamp: "Formatted at 2026-09-28 14:51:37 on dist-test-slave-vk40"
server_key: "9e64ca449b57fd2b734843af0fd752c4"
server_key_iv: "bdbd3ba12bc40543a36e661db82d7aba"
server_key_version: "encryptionkey@0"
I20260928 14:51:37.779486 25723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:51:37.782819 25738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:37.784173 25723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:51:37.784345 25723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d"
format_stamp: "Formatted at 2026-09-28 14:51:37 on dist-test-slave-vk40"
server_key: "9e64ca449b57fd2b734843af0fd752c4"
server_key_iv: "bdbd3ba12bc40543a36e661db82d7aba"
server_key_version: "encryptionkey@0"
I20260928 14:51:37.784519 25723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:37.798707 25723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:37.802965 25723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:37.803251 25723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:37.804194 25723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:37.805526 25723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:37.805600 25723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:37.805719 25723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:37.805775 25723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:37.835476 25723 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:33049
I20260928 14:51:37.835513 25850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:33049 every 8 connection(s)
I20260928 14:51:37.836752 25723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:51:37.839498 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 25723
I20260928 14:51:37.839718 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:51:37.840519 15303 external_mini_cluster.cc:1428] Setting key b44ee06eb17dd7015962698525fd78ee
Sep 28 14:51:37 dist-test-slave-vk40 krb5kdc[25617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607097, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:51:37.856628 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:37.839170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:42603 (local address 127.14.241.254:46089)
0928 14:51:37.839618 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:37.839626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:37.840885 (+ 1259us) server_negotiation.cc:407] Connection header received
0928 14:51:37.842923 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:37.842929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:37.843008 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:37.843099 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:37.844722 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:37.845454 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:37.846990 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:37.847398 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:37.847506 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:37.851056 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:37.851061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:37.851064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:37.851098 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:37.853510 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:37.854262 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:37.854266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:37.854268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:37.854391 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:37.854802 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:37.854805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:37.854807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:37.855095 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:37.855227 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:37.855978 (+ 751us) server_negotiation.cc:299] Negotiation successful
0928 14:51:37.856222 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":227,"threads_started":1}
I20260928 14:51:37.857712 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:37.839886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46089 (local address 127.14.241.193:42603)
0928 14:51:37.840715 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:37.840755 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:37.842641 (+ 1886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:37.843380 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:37.843389 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:37.843898 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:37.844554 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:37.844575 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:37.845599 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:37.845603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:37.846854 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:37.846867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:37.847619 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:37.847626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:37.847862 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:37.848540 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:37.848564 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:37.850827 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:37.853669 (+ 2842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:37.853678 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:37.853697 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:37.854134 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:37.854494 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:37.854499 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:37.854503 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:37.854693 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:37.855226 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:37.855234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:37.855608 (+ 374us) client_negotiation.cc:769] Sending connection context
0928 14:51:37.855876 (+ 268us) client_negotiation.cc:240] Negotiation successful
0928 14:51:37.856480 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":328,"threads_started":1}
I20260928 14:51:37.859222 25851 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46089
I20260928 14:51:37.859583 25851 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:37.860538 25851 heartbeater.cc:511] Master 127.14.241.254:46089 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
I20260928 14:51:37.862676 25662 ts_manager.cc:194] Registered new tserver with Master: e7f1442f3d2b4d2e9e81f6f250a5089d (127.14.241.193:33049)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
I20260928 14:51:37.864183 25662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:42603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:51:37.889619 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46089
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:38.212471 25857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:38.212772 25857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:38.212869 25857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:38.220407 25857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:38.220520 25857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:38.220639 25857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:51:38.230686 25857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:38.231931 25857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:38.233214 25857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:38.247113 25862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:38.247176 25863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:38.247165 25865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:38.249673 25857 server_base.cc:1034] running on GCE node
I20260928 14:51:38.250320 25857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:38.251298 25857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:38.252497 25857 hybrid_clock.cc:648] HybridClock initialized: now 1790607098252441 us; error 52 us; skew 500 ppm
I20260928 14:51:38.252815 25857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:38 dist-test-slave-vk40 krb5kdc[25617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607098, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:38.255916 25857 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:51:38.257462 25857 webserver.cc:460] Webserver started at http://127.14.241.194:45087/ using document root <none> and password file <none>
I20260928 14:51:38.258183 25857 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:38.258298 25857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:38.258589 25857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:38.261502 25857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "a0693506926a47578ea32d018bfaf2c3"
format_stamp: "Formatted at 2026-09-28 14:51:38 on dist-test-slave-vk40"
server_key: "5e1705993ec89c70229bb243d2a8ec30"
server_key_iv: "50cd0d446ba9523ff5decd0477d8bea6"
server_key_version: "encryptionkey@0"
I20260928 14:51:38.262259 25857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "a0693506926a47578ea32d018bfaf2c3"
format_stamp: "Formatted at 2026-09-28 14:51:38 on dist-test-slave-vk40"
server_key: "5e1705993ec89c70229bb243d2a8ec30"
server_key_iv: "50cd0d446ba9523ff5decd0477d8bea6"
server_key_version: "encryptionkey@0"
I20260928 14:51:38.267334 25857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260928 14:51:38.270967 25872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:38.272594 25857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:51:38.272778 25857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "a0693506926a47578ea32d018bfaf2c3"
format_stamp: "Formatted at 2026-09-28 14:51:38 on dist-test-slave-vk40"
server_key: "5e1705993ec89c70229bb243d2a8ec30"
server_key_iv: "50cd0d446ba9523ff5decd0477d8bea6"
server_key_version: "encryptionkey@0"
I20260928 14:51:38.272976 25857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:38.298581 25857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:38.303155 25857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:38.303458 25857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:38.304498 25857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:38.305883 25857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:38.305960 25857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:38.306077 25857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:38.306126 25857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:38.336313 25857 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:35967
I20260928 14:51:38.336380 25984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:35967 every 8 connection(s)
I20260928 14:51:38.337628 25857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:51:38.342001 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 25857
I20260928 14:51:38.342202 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:51:38.342710 15303 external_mini_cluster.cc:1428] Setting key 743d2fb314e2b65a08b19869f882c61a
Sep 28 14:51:38 dist-test-slave-vk40 krb5kdc[25617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607098, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:51:38.360378 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:38.340142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:54713 (local address 127.14.241.254:46089)
0928 14:51:38.340398 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:38.340404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:38.341584 (+ 1180us) server_negotiation.cc:407] Connection header received
0928 14:51:38.343875 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:38.343879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:38.343952 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:38.344040 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:38.346027 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.346791 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.349107 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.349558 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.349685 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:38.353942 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:38.353949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:38.353954 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:38.354000 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:38.357055 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:38.357943 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:38.357949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:38.357952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:38.358107 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:38.358535 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:38.358540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:38.358543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:38.358939 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:38.359070 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:38.359792 (+ 722us) server_negotiation.cc:299] Negotiation successful
0928 14:51:38.360040 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260928 14:51:38.361284 25987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:38.340723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46089 (local address 127.14.241.194:54713)
0928 14:51:38.341420 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:38.341459 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:38.343622 (+ 2163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:38.344339 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:38.344352 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:38.344982 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:38.345846 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:38.345872 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.346935 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.346940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:38.348928 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:38.348943 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.349787 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.349793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:38.350072 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:38.350843 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:38.350871 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:38.353632 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:38.357227 (+ 3595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:38.357237 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:38.357257 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:38.357792 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:38.358200 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:38.358203 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:38.358205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:38.358422 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:38.359060 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:38.359066 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:38.359423 (+ 357us) client_negotiation.cc:769] Sending connection context
0928 14:51:38.359654 (+ 231us) client_negotiation.cc:240] Negotiation successful
0928 14:51:38.360064 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":259,"threads_started":1}
I20260928 14:51:38.362819 25985 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46089
I20260928 14:51:38.363139 25985 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:38.363897 25985 heartbeater.cc:511] Master 127.14.241.254:46089 requested a full tablet report, sending...
I20260928 14:51:38.365598 25662 ts_manager.cc:194] Registered new tserver with Master: a0693506926a47578ea32d018bfaf2c3 (127.14.241.194:35967)
I20260928 14:51:38.366434 25662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:54713
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:51:38.396247 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46089
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:51:38.714224 25990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:51:38.714587 25990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:51:38.714744 25990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:51:38.722188 25990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:51:38.722299 25990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:51:38.722445 25990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:51:38.732138 25990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:51:38.733410 25990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:51:38.734696 25990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:51:38.747213 25996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:38.747291 25995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:51:38.747233 25998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:51:38.747985 25990 server_base.cc:1034] running on GCE node
I20260928 14:51:38.748574 25990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:51:38.749451 25990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:51:38.750629 25990 hybrid_clock.cc:648] HybridClock initialized: now 1790607098750577 us; error 49 us; skew 500 ppm
I20260928 14:51:38.750914 25990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:51:38 dist-test-slave-vk40 krb5kdc[25617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607098, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:51:38.753876 25990 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:51:38.755291 25990 webserver.cc:460] Webserver started at http://127.14.241.195:45785/ using document root <none> and password file <none>
I20260928 14:51:38.755923 25990 fs_manager.cc:359] Metadata directory not provided
I20260928 14:51:38.756026 25990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:51:38.756268 25990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:51:38.759265 25990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "53cd840c6d334f23bb6328fc999c4c17"
format_stamp: "Formatted at 2026-09-28 14:51:38 on dist-test-slave-vk40"
server_key: "c63248a280d6294a399e3f842e0507c1"
server_key_iv: "40a720655ac6e6df9380580fb83cd125"
server_key_version: "encryptionkey@0"
I20260928 14:51:38.760139 25990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "53cd840c6d334f23bb6328fc999c4c17"
format_stamp: "Formatted at 2026-09-28 14:51:38 on dist-test-slave-vk40"
server_key: "c63248a280d6294a399e3f842e0507c1"
server_key_iv: "40a720655ac6e6df9380580fb83cd125"
server_key_version: "encryptionkey@0"
I20260928 14:51:38.765796 25990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:51:38.769234 26005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:38.770537 25990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 14:51:38.770677 25990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "53cd840c6d334f23bb6328fc999c4c17"
format_stamp: "Formatted at 2026-09-28 14:51:38 on dist-test-slave-vk40"
server_key: "c63248a280d6294a399e3f842e0507c1"
server_key_iv: "40a720655ac6e6df9380580fb83cd125"
server_key_version: "encryptionkey@0"
I20260928 14:51:38.770861 25990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:51:38.794085 25990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:51:38.798218 25990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:51:38.798498 25990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:51:38.799470 25990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:51:38.800832 25990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:51:38.800906 25990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:38.801013 25990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:51:38.801060 25990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:51:38.830542 25990 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:41099
I20260928 14:51:38.830578 26117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:41099 every 8 connection(s)
I20260928 14:51:38.831791 25990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:51:38.838223 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 25990
I20260928 14:51:38.838405 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:51:38.838868 15303 external_mini_cluster.cc:1428] Setting key ec186288aafc036013b415ae042f2deb
Sep 28 14:51:38 dist-test-slave-vk40 krb5kdc[25617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607098, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:51:38.850579 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:38.834149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:34469 (local address 127.14.241.254:46089)
0928 14:51:38.834332 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:38.834337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:38.835540 (+ 1203us) server_negotiation.cc:407] Connection header received
0928 14:51:38.837405 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:38.837412 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:38.837513 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:38.837638 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:38.839178 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.840123 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.842130 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.842541 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.842634 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:38.845702 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:38.845707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:38.845710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:38.845745 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:38.848103 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:38.848783 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:38.848786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:38.848788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:38.848911 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:38.849230 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:38.849233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:38.849235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:38.849488 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:38.849590 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:38.850172 (+ 582us) server_negotiation.cc:299] Negotiation successful
0928 14:51:38.850333 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:51:38.851528 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:38.834584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46089 (local address 127.14.241.195:34469)
0928 14:51:38.835350 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:38.835410 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:38.837134 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:38.837799 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:38.837807 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:38.838288 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:51:38.838949 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:38.838979 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.840311 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.840317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:38.842010 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:38.842024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.842698 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.842703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:38.842890 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:38.843428 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:51:38.843448 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:51:38.845523 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:51:38.848230 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:38.848236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:38.848248 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:38.848691 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:38.848987 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:51:38.848990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:51:38.848992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:51:38.849154 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:51:38.849572 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:51:38.849578 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:51:38.849883 (+ 305us) client_negotiation.cc:769] Sending connection context
0928 14:51:38.850098 (+ 215us) client_negotiation.cc:240] Negotiation successful
0928 14:51:38.850483 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":324,"threads_started":1}
I20260928 14:51:38.852910 26118 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46089
I20260928 14:51:38.853200 26118 heartbeater.cc:464] Registering TS with master...
I20260928 14:51:38.853993 26118 heartbeater.cc:511] Master 127.14.241.254:46089 requested a full tablet report, sending...
I20260928 14:51:38.855261 25662 ts_manager.cc:194] Registered new tserver with Master: 53cd840c6d334f23bb6328fc999c4c17 (127.14.241.195:41099)
I20260928 14:51:38.856029 25662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:34469
I20260928 14:51:38.866118 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:51:38.867087 25851 heartbeater.cc:503] Master 127.14.241.254:46089 was elected leader, sending a full tablet report...
I20260928 14:51:38.882007 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:38.870641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.14.241.254:46089)
0928 14:51:38.870788 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:38.870793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:38.871237 (+ 444us) server_negotiation.cc:407] Connection header received
0928 14:51:38.871356 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:38.871360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:38.871426 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:38.871506 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:38.872822 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.873556 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.874760 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.875286 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.875374 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:38.877575 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:38.877580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:38.877583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:38.877614 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:38.879826 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:38.880420 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:38.880423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:38.880425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:38.880548 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:38.880829 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:38.880832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:38.880834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:38.881177 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:38.881349 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:38.881617 (+ 268us) server_negotiation.cc:299] Negotiation successful
0928 14:51:38.881775 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:51:38.885793 25662 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:51:38.892570 25662 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:51:38.925964 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:38.906612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:35967 (local address 127.0.0.1:40104)
0928 14:51:38.918377 (+ 11765us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:38.918412 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:38.918530 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:38.919127 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:38.919131 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:38.919156 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:38.919510 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:38.919521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.921187 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.921196 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:38.923456 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:38.923472 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.924957 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.924963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:38.925155 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:38.925204 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 14:51:38.925325 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 14:51:38.925517 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11633,"mutex_wait_us":30,"thread_start_us":5867,"threads_started":1}
I20260928 14:51:38.926080 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:38.904553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34292 (local address 127.14.241.195:41099)
0928 14:51:38.916548 (+ 11995us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:38.916559 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:38.916604 (+ 45us) server_negotiation.cc:407] Connection header received
0928 14:51:38.916715 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:38.916724 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:38.917031 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:38.917349 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:38.918705 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.920329 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.923292 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.924645 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.924805 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:38.925000 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:38.925569 (+ 569us) server_negotiation.cc:299] Negotiation successful
0928 14:51:38.925731 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11746,"thread_start_us":11552,"threads_started":1}
I20260928 14:51:38.926754 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:38.903732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33049 (local address 127.0.0.1:57854)
0928 14:51:38.912955 (+ 9223us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:38.913023 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:38.913228 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:38.919381 (+ 6153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:38.919387 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:38.919415 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:38.919837 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:38.919850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.921705 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.921710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:38.923908 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:38.923922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.925906 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.925912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:38.926082 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:38.926113 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 14:51:38.926202 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 14:51:38.926333 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8912,"mutex_wait_us":26,"thread_start_us":14127,"threads_started":1}
I20260928 14:51:38.926970 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:38.903938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:41099 (local address 127.0.0.1:34292)
0928 14:51:38.914851 (+ 10913us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:38.914875 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:38.915001 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:38.917299 (+ 2298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:38.917308 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:38.917344 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:38.917988 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:38.918007 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.920552 (+ 2545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.920558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:38.923049 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:38.923066 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.925185 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.925191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:38.925363 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:38.925397 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 14:51:38.926385 (+ 988us) client_negotiation.cc:240] Negotiation successful
0928 14:51:38.926510 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10775,"mutex_wait_us":51,"thread_start_us":10622,"threads_started":1}
I20260928 14:51:38.927022 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:38.906708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.14.241.194:35967)
0928 14:51:38.918270 (+ 11562us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:38.918280 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:38.918506 (+ 226us) server_negotiation.cc:407] Connection header received
0928 14:51:38.918653 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:38.918661 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:38.918922 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:38.919118 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:38.919686 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.921071 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.923647 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.924770 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.924919 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:38.925094 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:38.925383 (+ 289us) server_negotiation.cc:299] Negotiation successful
0928 14:51:38.925551 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11387,"thread_start_us":11294,"threads_started":1}
I20260928 14:51:38.927460 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:38.904019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57854 (local address 127.14.241.193:33049)
0928 14:51:38.918684 (+ 14665us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:38.918692 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:38.918718 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:51:38.918791 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:38.918796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:38.918980 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:38.919145 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:38.920041 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.921548 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.924061 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:38.925434 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:38.925611 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:38.925755 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:38.927030 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0928 14:51:38.927192 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14496,"thread_start_us":14341,"threads_started":1}
I20260928 14:51:38.930398 25920 tablet_service.cc:1467] Processing CreateTablet for tablet 023671616f52470b85b5b5797d2aa540 (DEFAULT_TABLE table=test-table [id=c744ffadf61847198f4af4922961faa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:38.930752 26053 tablet_service.cc:1467] Processing CreateTablet for tablet 023671616f52470b85b5b5797d2aa540 (DEFAULT_TABLE table=test-table [id=c744ffadf61847198f4af4922961faa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:38.930753 25786 tablet_service.cc:1467] Processing CreateTablet for tablet 023671616f52470b85b5b5797d2aa540 (DEFAULT_TABLE table=test-table [id=c744ffadf61847198f4af4922961faa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:51:38.931635 25920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 023671616f52470b85b5b5797d2aa540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:38.932127 26053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 023671616f52470b85b5b5797d2aa540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:38.932128 25786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 023671616f52470b85b5b5797d2aa540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:51:38.944227 26136 tablet_bootstrap.cc:492] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: Bootstrap starting.
I20260928 14:51:38.946542 26137 tablet_bootstrap.cc:492] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: Bootstrap starting.
I20260928 14:51:38.947942 26136 tablet_bootstrap.cc:654] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:38.948166 26138 tablet_bootstrap.cc:492] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: Bootstrap starting.
I20260928 14:51:38.949375 26136 log.cc:824] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: Log is configured to *not* fsync() on all Append() calls
I20260928 14:51:38.949928 26137 tablet_bootstrap.cc:654] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:38.950945 26137 log.cc:824] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: Log is configured to *not* fsync() on all Append() calls
I20260928 14:51:38.951052 26138 tablet_bootstrap.cc:654] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: Neither blocks nor log segments found. Creating new log.
I20260928 14:51:38.952178 26138 log.cc:824] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:51:38.952433 26136 tablet_bootstrap.cc:492] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: No bootstrap required, opened a new log
I20260928 14:51:38.952698 26136 ts_tablet_manager.cc:1397] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260928 14:51:38.953965 26137 tablet_bootstrap.cc:492] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: No bootstrap required, opened a new log
I20260928 14:51:38.954216 26137 ts_tablet_manager.cc:1397] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260928 14:51:38.954855 26138 tablet_bootstrap.cc:492] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: No bootstrap required, opened a new log
I20260928 14:51:38.955106 26138 ts_tablet_manager.cc:1397] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260928 14:51:38.962028 26138 raft_consensus.cc:348] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33049 } } peers { permanent_uuid: "53cd840c6d334f23bb6328fc999c4c17" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 41099 } } peers { permanent_uuid: "a0693506926a47578ea32d018bfaf2c3" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 35967 } }
I20260928 14:51:38.962404 26138 raft_consensus.cc:374] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:38.962496 26138 raft_consensus.cc:729] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7f1442f3d2b4d2e9e81f6f250a5089d, State: Initialized, Role: FOLLOWER
I20260928 14:51:38.963088 26138 consensus_queue.cc:260] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33049 } } peers { permanent_uuid: "53cd840c6d334f23bb6328fc999c4c17" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 41099 } } peers { permanent_uuid: "a0693506926a47578ea32d018bfaf2c3" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 35967 } }
I20260928 14:51:38.963698 26136 raft_consensus.cc:348] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33049 } } peers { permanent_uuid: "53cd840c6d334f23bb6328fc999c4c17" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 41099 } } peers { permanent_uuid: "a0693506926a47578ea32d018bfaf2c3" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 35967 } }
I20260928 14:51:38.964180 26136 raft_consensus.cc:374] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:38.964319 26136 raft_consensus.cc:729] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0693506926a47578ea32d018bfaf2c3, State: Initialized, Role: FOLLOWER
I20260928 14:51:38.964942 26138 ts_tablet_manager.cc:1428] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260928 14:51:38.964952 26136 consensus_queue.cc:260] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33049 } } peers { permanent_uuid: "53cd840c6d334f23bb6328fc999c4c17" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 41099 } } peers { permanent_uuid: "a0693506926a47578ea32d018bfaf2c3" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 35967 } }
I20260928 14:51:38.965095 26137 raft_consensus.cc:348] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33049 } } peers { permanent_uuid: "53cd840c6d334f23bb6328fc999c4c17" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 41099 } } peers { permanent_uuid: "a0693506926a47578ea32d018bfaf2c3" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 35967 } }
I20260928 14:51:38.965541 26137 raft_consensus.cc:374] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:51:38.965642 26137 raft_consensus.cc:729] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53cd840c6d334f23bb6328fc999c4c17, State: Initialized, Role: FOLLOWER
I20260928 14:51:38.966218 26137 consensus_queue.cc:260] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33049 } } peers { permanent_uuid: "53cd840c6d334f23bb6328fc999c4c17" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 41099 } } peers { permanent_uuid: "a0693506926a47578ea32d018bfaf2c3" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 35967 } }
I20260928 14:51:38.966733 25985 heartbeater.cc:503] Master 127.14.241.254:46089 was elected leader, sending a full tablet report...
I20260928 14:51:38.967881 26137 ts_tablet_manager.cc:1428] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260928 14:51:38.968222 26118 heartbeater.cc:503] Master 127.14.241.254:46089 was elected leader, sending a full tablet report...
I20260928 14:51:38.970137 26136 ts_tablet_manager.cc:1428] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: Time spent starting tablet: real 0.017s user 0.010s sys 0.003s
I20260928 14:51:39.010766 26144 raft_consensus.cc:482] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:51:39.011055 26144 raft_consensus.cc:504] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33049 } } peers { permanent_uuid: "53cd840c6d334f23bb6328fc999c4c17" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 41099 } } peers { permanent_uuid: "a0693506926a47578ea32d018bfaf2c3" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 35967 } }
I20260928 14:51:39.012804 26144 leader_election.cc:288] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7f1442f3d2b4d2e9e81f6f250a5089d (127.14.241.193:33049), a0693506926a47578ea32d018bfaf2c3 (127.14.241.194:35967)
I20260928 14:51:39.018750 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:39.013000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33049 (local address 127.14.241.195:55353)
0928 14:51:39.013216 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:39.013235 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:39.013360 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:39.013762 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:39.013766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:39.013794 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:39.013978 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:39.013990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.015158 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.015164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:39.016683 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:39.016695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.018060 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.018066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:39.018219 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:39.018244 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 14:51:39.018319 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 14:51:39.018442 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":1}
I20260928 14:51:39.018733 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:39.013116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:55353 (local address 127.14.241.193:33049)
0928 14:51:39.013310 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:39.013317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:39.013342 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:51:39.013549 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:39.013555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:39.013632 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:39.013717 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:39.014128 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.015032 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.016830 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.017887 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.018023 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:39.018096 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:39.018372 (+ 276us) server_negotiation.cc:299] Negotiation successful
0928 14:51:39.018490 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 14:51:39.019346 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:39.013372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:48277 (local address 127.14.241.194:35967)
0928 14:51:39.013596 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:39.013601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:39.013964 (+ 363us) server_negotiation.cc:407] Connection header received
0928 14:51:39.014094 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:39.014099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:39.014162 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:39.014225 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:51:39.014735 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.015656 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.017600 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.018450 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.018589 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:39.018657 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:39.018986 (+ 329us) server_negotiation.cc:299] Negotiation successful
0928 14:51:39.019119 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260928 14:51:39.019968 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:39.013114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:35967 (local address 127.14.241.195:48277)
0928 14:51:39.013862 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0928 14:51:39.013875 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:51:39.013984 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:51:39.014261 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:51:39.014265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:51:39.014288 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:51:39.014602 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:39.014613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.015791 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.015795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:39.017404 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:51:39.017418 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.018650 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.018656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:51:39.018799 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:39.018830 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 14:51:39.019535 (+ 705us) client_negotiation.cc:240] Negotiation successful
0928 14:51:39.019667 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":406,"threads_started":1}
I20260928 14:51:39.019783 25806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023671616f52470b85b5b5797d2aa540" candidate_uuid: "53cd840c6d334f23bb6328fc999c4c17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" is_pre_election: true
I20260928 14:51:39.020349 25806 raft_consensus.cc:2454] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53cd840c6d334f23bb6328fc999c4c17 in term 0.
I20260928 14:51:39.020888 25940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023671616f52470b85b5b5797d2aa540" candidate_uuid: "53cd840c6d334f23bb6328fc999c4c17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0693506926a47578ea32d018bfaf2c3" is_pre_election: true
I20260928 14:51:39.021001 26009 leader_election.cc:302] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53cd840c6d334f23bb6328fc999c4c17, e7f1442f3d2b4d2e9e81f6f250a5089d; no voters:
I20260928 14:51:39.021409 26144 raft_consensus.cc:2781] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:51:39.021410 25940 raft_consensus.cc:2454] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53cd840c6d334f23bb6328fc999c4c17 in term 0.
I20260928 14:51:39.021552 26144 raft_consensus.cc:482] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:51:39.021651 26144 raft_consensus.cc:3037] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:39.024011 26144 raft_consensus.cc:504] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33049 } } peers { permanent_uuid: "53cd840c6d334f23bb6328fc999c4c17" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 41099 } } peers { permanent_uuid: "a0693506926a47578ea32d018bfaf2c3" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 35967 } }
I20260928 14:51:39.024636 26144 leader_election.cc:288] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [CANDIDATE]: Term 1 election: Requested vote from peers e7f1442f3d2b4d2e9e81f6f250a5089d (127.14.241.193:33049), a0693506926a47578ea32d018bfaf2c3 (127.14.241.194:35967)
I20260928 14:51:39.025075 25940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023671616f52470b85b5b5797d2aa540" candidate_uuid: "53cd840c6d334f23bb6328fc999c4c17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0693506926a47578ea32d018bfaf2c3"
I20260928 14:51:39.025059 25806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "023671616f52470b85b5b5797d2aa540" candidate_uuid: "53cd840c6d334f23bb6328fc999c4c17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d"
I20260928 14:51:39.025264 25940 raft_consensus.cc:3037] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:39.025264 25806 raft_consensus.cc:3037] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:51:39.028403 25806 raft_consensus.cc:2454] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53cd840c6d334f23bb6328fc999c4c17 in term 1.
I20260928 14:51:39.028411 25940 raft_consensus.cc:2454] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53cd840c6d334f23bb6328fc999c4c17 in term 1.
I20260928 14:51:39.028862 26007 leader_election.cc:302] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53cd840c6d334f23bb6328fc999c4c17, a0693506926a47578ea32d018bfaf2c3; no voters:
I20260928 14:51:39.029219 26144 raft_consensus.cc:2781] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:51:39.029661 26144 raft_consensus.cc:686] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 1 LEADER]: Becoming Leader. State: Replica: 53cd840c6d334f23bb6328fc999c4c17, State: Running, Role: LEADER
I20260928 14:51:39.030109 26144 consensus_queue.cc:237] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33049 } } peers { permanent_uuid: "53cd840c6d334f23bb6328fc999c4c17" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 41099 } } peers { permanent_uuid: "a0693506926a47578ea32d018bfaf2c3" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 35967 } }
I20260928 14:51:39.034519 25661 catalog_manager.cc:5495] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53cd840c6d334f23bb6328fc999c4c17 (127.14.241.195). New cstate: current_term: 1 leader_uuid: "53cd840c6d334f23bb6328fc999c4c17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53cd840c6d334f23bb6328fc999c4c17" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 41099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0693506926a47578ea32d018bfaf2c3" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 35967 } health_report { overall_health: UNKNOWN } } }
I20260928 14:51:39.070727 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:39.065913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34300 (local address 127.14.241.195:41099)
0928 14:51:39.066098 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:39.066103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:39.066121 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:51:39.066252 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:39.066256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:39.066345 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:39.066473 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:51:39.066795 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.067494 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.068731 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.069240 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.069329 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:39.069736 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:51:39.070238 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:51:39.070348 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:39.070405 (+ 57us) server_negotiation.cc:299] Negotiation successful
0928 14:51:39.070472 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 14:51:39.078655 25806 raft_consensus.cc:1261] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d [term 1 FOLLOWER]: Refusing update from remote peer 53cd840c6d334f23bb6328fc999c4c17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:39.078655 25940 raft_consensus.cc:1261] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3 [term 1 FOLLOWER]: Refusing update from remote peer 53cd840c6d334f23bb6328fc999c4c17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:51:39.079623 26146 consensus_queue.cc:1035] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7f1442f3d2b4d2e9e81f6f250a5089d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:51:39.079955 26144 consensus_queue.cc:1035] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0693506926a47578ea32d018bfaf2c3" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 35967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 14:51:39.083244 26119 tablet.cc:2369] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:51:39.088826 25852 tablet.cc:2369] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:51:39.089740 25986 tablet.cc:2369] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:51:39.091009 26149 mvcc.cc:205] Tried to move back new op lower bound from 7334326677816758272 to 7334326677631070208. Current Snapshot: MvccSnapshot[applied={T|T < 7334326677816758272}]
I20260928 14:51:39.091343 26150 mvcc.cc:205] Tried to move back new op lower bound from 7334326677816758272 to 7334326677631070208. Current Snapshot: MvccSnapshot[applied={T|T < 7334326677816758272}]
I20260928 14:51:39.104177 25662 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:39.104485 25662 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:51:39.107769 25662 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b71ff603f824876a418b14ebf3a531f: Sending DeleteTablet for 3 replicas of tablet 023671616f52470b85b5b5797d2aa540
I20260928 14:51:39.108891 25920 tablet_service.cc:1514] Processing DeleteTablet for tablet 023671616f52470b85b5b5797d2aa540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:39 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:40104
I20260928 14:51:39.108898 26053 tablet_service.cc:1514] Processing DeleteTablet for tablet 023671616f52470b85b5b5797d2aa540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:39 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:34292
I20260928 14:51:39.108953 25786 tablet_service.cc:1514] Processing DeleteTablet for tablet 023671616f52470b85b5b5797d2aa540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:51:39 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:57854
I20260928 14:51:39.111270 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 25723
I20260928 14:51:39.122952 26159 tablet_replica.cc:330] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: stopping tablet replica
I20260928 14:51:39.123500 26159 raft_consensus.cc:2229] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:39.125015 26159 raft_consensus.cc:2258] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:39.128113 26159 ts_tablet_manager.cc:1905] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:39.133503 26158 tablet_replica.cc:330] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: stopping tablet replica
I20260928 14:51:39.134003 26158 raft_consensus.cc:2229] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:51:39.134698 26158 raft_consensus.cc:2258] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:39.136879 26160 tablet_replica.cc:330] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: stopping tablet replica
I20260928 14:51:39.137311 26160 raft_consensus.cc:2229] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:51:39.137588 26158 ts_tablet_manager.cc:1905] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:51:39.137821 26160 raft_consensus.cc:2258] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:51:39.140025 26160 ts_tablet_manager.cc:1905] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 14:51:39 dist-test-slave-vk40 krb5kdc[25617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:51:39.142920 26159 ts_tablet_manager.cc:1918] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:39.143088 26159 log.cc:1196] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/023671616f52470b85b5b5797d2aa540
I20260928 14:51:39.143611 26159 ts_tablet_manager.cc:1939] T 023671616f52470b85b5b5797d2aa540 P a0693506926a47578ea32d018bfaf2c3: Deleting consensus metadata
I20260928 14:51:39.145623 25647 catalog_manager.cc:4841] TS a0693506926a47578ea32d018bfaf2c3 (127.14.241.194:35967): tablet 023671616f52470b85b5b5797d2aa540 (table test-table [id=c744ffadf61847198f4af4922961faa3]) successfully deleted
I20260928 14:51:39.147879 26160 ts_tablet_manager.cc:1918] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:39.148058 26160 log.cc:1196] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/023671616f52470b85b5b5797d2aa540
I20260928 14:51:39.148563 26160 ts_tablet_manager.cc:1939] T 023671616f52470b85b5b5797d2aa540 P e7f1442f3d2b4d2e9e81f6f250a5089d: Deleting consensus metadata
I20260928 14:51:39.150401 25649 catalog_manager.cc:4841] TS e7f1442f3d2b4d2e9e81f6f250a5089d (127.14.241.193:33049): tablet 023671616f52470b85b5b5797d2aa540 (table test-table [id=c744ffadf61847198f4af4922961faa3]) successfully deleted
I20260928 14:51:39.150519 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:39.112741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57862 (local address 127.14.241.193:33049)
0928 14:51:39.112935 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:39.112942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:39.129284 (+ 16342us) server_negotiation.cc:407] Connection header received
0928 14:51:39.129774 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:39.129780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:39.129867 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:39.129959 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:39.131717 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.132995 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.134714 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.135354 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.135452 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:39.141557 (+ 6105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:39.141575 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:39.141594 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:39.141649 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:39.146514 (+ 4865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:39.147537 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:39.147547 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:39.147551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:39.147723 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:39.148424 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:39.148429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:39.148433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:39.149014 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:39.149316 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:39.149920 (+ 604us) server_negotiation.cc:299] Negotiation successful
0928 14:51:39.150159 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:51:39.150836 26158 ts_tablet_manager.cc:1918] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:51:39.151024 26158 log.cc:1196] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/023671616f52470b85b5b5797d2aa540
I20260928 14:51:39.151449 26158 ts_tablet_manager.cc:1939] T 023671616f52470b85b5b5797d2aa540 P 53cd840c6d334f23bb6328fc999c4c17: Deleting consensus metadata
I20260928 14:51:39.152390 25846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:39.153172 25646 catalog_manager.cc:4841] TS 53cd840c6d334f23bb6328fc999c4c17 (127.14.241.195:41099): tablet 023671616f52470b85b5b5797d2aa540 (table test-table [id=c744ffadf61847198f4af4922961faa3]) successfully deleted
I20260928 14:51:39.302496 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 25723
I20260928 14:51:39.318071 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 25857
Sep 28 14:51:39 dist-test-slave-vk40 krb5kdc[25617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:51:39.334123 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:39.318940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40110 (local address 127.14.241.194:35967)
0928 14:51:39.319114 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:39.319120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:39.319142 (+ 22us) server_negotiation.cc:407] Connection header received
0928 14:51:39.319244 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:39.319251 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:39.319322 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:39.319415 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:39.320713 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.321554 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.322896 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.323357 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.323466 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:39.327776 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:39.327796 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:39.327818 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:39.327864 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:39.331361 (+ 3497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:39.332036 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:39.332045 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:39.332049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:39.332195 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:39.332595 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:39.332600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:39.332604 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:39.333073 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:39.333374 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:39.333524 (+ 150us) server_negotiation.cc:299] Negotiation successful
0928 14:51:39.333739 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 14:51:39.334859 25980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:39.422194 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 25857
I20260928 14:51:39.436463 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 25990
Sep 28 14:51:39 dist-test-slave-vk40 krb5kdc[25617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:51:39.454159 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:51:39.437171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34312 (local address 127.14.241.195:41099)
0928 14:51:39.437340 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:51:39.437345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:51:39.437393 (+ 48us) server_negotiation.cc:407] Connection header received
0928 14:51:39.437580 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:51:39.437588 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:51:39.437688 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:51:39.437784 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:51:39.439045 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.439866 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.441229 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:51:39.441935 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:51:39.442046 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:51:39.447176 (+ 5130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:51:39.447195 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:51:39.447219 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:51:39.447282 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:51:39.451124 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:39.451782 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:39.451792 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:39.451796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:39.451994 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:51:39.452458 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:51:39.452463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:51:39.452466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:51:39.452959 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:51:39.453287 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0928 14:51:39.453508 (+ 221us) server_negotiation.cc:299] Negotiation successful
0928 14:51:39.453750 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:51:39.454988 26113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:39.573992 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 25990
I20260928 14:51:39.590379 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 25630
I20260928 14:51:39.591485 25692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:51:39.699786 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 25630
2026-09-28T14:51:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4110 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:51:39 dist-test-slave-vk40 krb5kdc[26168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:51:39 dist-test-slave-vk40 krb5kdc[26168](info): set up 2 sockets
Sep 28 14:51:39 dist-test-slave-vk40 krb5kdc[26168](info): commencing operation
krb5kdc: starting...
W20260928 14:51:40.711774 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:51:40 dist-test-slave-vk40 krb5kdc[26168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:51:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:51:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/rangeradmin_127.14.241.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/rangeradmin_127.14.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/rangerlookup_127.14.241.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/rangerlookup_127.14.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/HTTP_127.14.241.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/HTTP_127.14.241.212@KRBTEST.COM.keytab.
I20260928 14:51:41.040966 15303 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKMueFh/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/postgres -l logfile start
2026-09-28 14:51:43.548 UTC [26198] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 14:51:43.548 UTC [26198] LOG: listening on IPv4 address "127.14.241.212", port 40031
2026-09-28 14:51:43.552 UTC [26198] LOG: listening on Unix socket "/tmp/.s.PGSQL.40031"
2026-09-28 14:51:43.574 UTC [26201] LOG: database system was shut down at 2026-09-28 14:51:42 UTC
2026-09-28 14:51:43.577 UTC [26198] LOG: database system is ready to accept connections
I20260928 14:51:44.471447 15303 mini_postgres.cc:96] Postgres bound to 40031
2026-09-28 14:51:44.488 UTC [26209] FATAL: database "slave" does not exist
127.14.241.212:40031 - accepting connections
I20260928 14:51:44.488973 15303 mini_ranger.cc:161] Starting Ranger...
I20260928 14:51:44.514637 15303 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 14:51:45.007305 15303 mini_ranger.cc:87] Created ranger Postgres database
I20260928 14:51:45.007417 15303 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-admin
2026-09-28 14:51:45,414 [I] DB FLAVOR :POSTGRES
2026-09-28 14:51:45,415 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:51:45,415 [I] Checking connection..
2026-09-28 14:51:45,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 14:51:45,664 [I] Checking connection passed.
2026-09-28 14:51:45,664 [I] --------- Verifying version history table ---------
2026-09-28 14:51:45,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:51:45,908 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:51:45,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:51:46,168 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:51:46,169 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 14:51:46,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 14:51:46.421 UTC [26273] WARNING: there is no transaction in progress
2026-09-28 14:51:46,427 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 14:51:46,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:51:46,676 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 14:51:46,676 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:51:46,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 14:51:46,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 14:51:47,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vk40.c.gcp-upstream.internal','N') ;"
2026-09-28 14:51:47,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:51:47,672 [I] Table x_portal_user does not exist in database ranger
2026-09-28 14:51:47,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:51:47,912 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 14:51:47,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:51:48,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:51:48,378 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 14:51:48,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 14:51:49.347 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.363 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.380 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.396 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.412 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.429 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.485 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.492 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.507 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.517 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.528 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.542 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.552 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.558 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.566 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:49.573 UTC [26381] WARNING: there is no transaction in progress
2026-09-28 14:51:50,232 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 14:51:50,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 14:51:50,460 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 14:51:50,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:51:50,701 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 14:51:50,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:51:50,968 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 14:51:50,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:51:51,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:51:51,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vk40.c.gcp-upstream.internal';"
2026-09-28 14:51:51,689 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 14:51:51,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 14:51:51,924 [I] DB_PATCHES have already been applied
I20260928 14:51:51.930079 15303 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKMueFh/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 14:51:51.930166 15303 mini_ranger.cc:193] Using host: 127.14.241.212
I20260928 14:51:51.944522 15303 mini_ranger.cc:227] Ranger admin URL: http://127.14.241.212:56823
Sep 28, 2026 2:51:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:51:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:51:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 2:51:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 2:51:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 2:51:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 2:51:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 14:51:52.971585 15303 mini_ranger.cc:160] Time spent starting Ranger: real 8.483s user 0.007s sys 0.033s
Sep 28, 2026 2:52:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 2:52:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 2:52:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 2:52:13 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 2:52:14 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 14:52:15.475561 15303 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/rangerkms_127.14.241.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/rangerkms_127.14.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.241.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.241.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.241.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/HTTP_127.14.241.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.241.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/HTTP_127.14.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 14:52:15 dist-test-slave-vk40 krb5kdc[26168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607135, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 14:52:15.663585 15303 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 14:52:15.687673 15303 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 14:52:16.561944 15303 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 14:52:16.562103 15303 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-kms
I20260928 14:52:16.562247 15303 mini_ranger_kms.cc:227] Using postgres at 127.14.241.212:40031
2026-09-28 14:52:17,053 [I] DB FLAVOR :POSTGRES
2026-09-28 14:52:17,053 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:52:17,053 [I] Checking connection
2026-09-28 14:52:17,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 14:52:17,303 [I] connection success
2026-09-28 14:52:17,303 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 14:52:17,304 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 14:52:17,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 14:52:17,552 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 14:52:17,552 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:52:17,553 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 14:52:17,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:40031/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 14:52:17,828 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 14:52:18.301553 15303 mini_ranger_kms.cc:310] Created kms service
I20260928 14:52:18.478085 15303 mini_ranger_kms.cc:326] Created kudu user
I20260928 14:52:18.522583 15303 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 14:52:18 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607135, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:52:18.934363 15303 mini_ranger_kms.cc:384] Added ranger policy
I20260928 14:52:18.934672 15303 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 14:52:18.934720 15303 mini_ranger_kms.cc:240] Using host: 127.14.241.212
I20260928 14:52:18.960242 15303 mini_ranger_kms.cc:276] Ranger KMS PID: 26562
I20260928 14:52:18.960444 15303 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.241.212:57763
Sep 28, 2026 2:52:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:52:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:52:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 2:52:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 2:52:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 2:52:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 14:52:21.080312 15303 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.417s user 0.169s sys 0.280s
I20260928 14:52:21.080593 15303 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 14:52:21.080803 15303 mini_ranger_kms.cc:401] 127.14.241.212:57763/kms/v1/keys
Sep 28 14:52:21 dist-test-slave-vk40 krb5kdc[26168](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790607141, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.241.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 14:52:21 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607141, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.241.212@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
Sep 28 14:52:21 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607141, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.241.212@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
Sep 28 14:52:24 dist-test-slave-vk40 krb5kdc[26168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:52:24.264124 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41345
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:41345
--ranger_config_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:57763/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:52:24.625090 26612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:52:24.625392 26612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:52:24.625509 26612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:52:24.634263 26612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:52:24.634438 26612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:52:24.634527 26612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:52:24.634603 26612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:52:24.634665 26612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:52:24.645836 26612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.241.212:57763/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:41345
--ranger_config_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:52:24.647125 26612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:52:24.648412 26612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:52:24.659711 26617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:52:24.659930 26618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:52:24.659744 26620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:52:24.665818 26612 server_base.cc:1034] running on GCE node
I20260928 14:52:24.666649 26612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:52:24.668119 26612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:52:24.669333 26612 hybrid_clock.cc:648] HybridClock initialized: now 1790607144669278 us; error 51 us; skew 500 ppm
I20260928 14:52:24.669680 26612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:52:24 dist-test-slave-vk40 krb5kdc[26168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607144, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:52:24.672696 26612 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:52:24.674209 26612 webserver.cc:460] Webserver started at http://127.14.241.254:33649/ using document root <none> and password file <none>
I20260928 14:52:24.675047 26612 fs_manager.cc:359] Metadata directory not provided
I20260928 14:52:24.675170 26612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:52:24.675469 26612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:52:24 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607144, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:52:24.739442 26612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "f51376f2af254cbdaaf0a518f86631d7"
format_stamp: "Formatted at 2026-09-28 14:52:24 on dist-test-slave-vk40"
server_key: "27df7f28f367470773a4608ab1886393"
server_key_iv: "c106732b572df219f7b958f036c4862d"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:24.740329 26612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "f51376f2af254cbdaaf0a518f86631d7"
format_stamp: "Formatted at 2026-09-28 14:52:24 on dist-test-slave-vk40"
server_key: "27df7f28f367470773a4608ab1886393"
server_key_iv: "c106732b572df219f7b958f036c4862d"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:24.745596 26612 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:52:24.766125 26627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:52:24.767661 26612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:52:24.767832 26612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "f51376f2af254cbdaaf0a518f86631d7"
format_stamp: "Formatted at 2026-09-28 14:52:24 on dist-test-slave-vk40"
server_key: "27df7f28f367470773a4608ab1886393"
server_key_iv: "c106732b572df219f7b958f036c4862d"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:24.768056 26612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:52:24.787147 26612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:52:24.791016 26612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:52:24.791294 26612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:52:24.807828 26612 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:41345
I20260928 14:52:24.807883 26678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:41345 every 8 connection(s)
I20260928 14:52:24.809181 26612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:52:24.811137 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 26612
I20260928 14:52:24.811352 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:52:24.812131 15303 external_mini_cluster.cc:1428] Setting key 0df55502d94d6d2d598e4aa09ba249b9
I20260928 14:52:24.813673 26679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 14:52:24 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:52:24.825008 26679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7: Bootstrap starting.
I20260928 14:52:24.828366 26679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7: Neither blocks nor log segments found. Creating new log.
I20260928 14:52:24.829500 26679 log.cc:824] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7: Log is configured to *not* fsync() on all Append() calls
I20260928 14:52:24.832408 26679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7: No bootstrap required, opened a new log
I20260928 14:52:24.832983 26682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:24.814192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.14.241.254:41345)
0928 14:52:24.814945 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:24.814957 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:24.814996 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:52:24.816593 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:24.816614 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:24.817007 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:24.817394 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:52:24.818645 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:24.820003 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:24.821328 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:24.822033 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:24.822212 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:24.826035 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:52:24.826042 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:52:24.826057 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:52:24.826094 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:52:24.828928 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:24.829734 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:24.829742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:24.829748 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:24.829911 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:24.830275 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:24.830280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:24.830283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:24.830823 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:52:24.831097 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:24.831516 (+ 419us) server_negotiation.cc:299] Negotiation successful
0928 14:52:24.831901 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":349,"threads_started":1}
I20260928 14:52:24.841777 26679 raft_consensus.cc:348] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f51376f2af254cbdaaf0a518f86631d7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41345 } }
I20260928 14:52:24.842257 26679 raft_consensus.cc:374] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:52:24.842358 26679 raft_consensus.cc:729] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51376f2af254cbdaaf0a518f86631d7, State: Initialized, Role: FOLLOWER
I20260928 14:52:24.842898 26679 consensus_queue.cc:260] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f51376f2af254cbdaaf0a518f86631d7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41345 } }
I20260928 14:52:24.843110 26679 raft_consensus.cc:388] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:52:24.843230 26679 raft_consensus.cc:482] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:52:24.843382 26679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:52:24.845600 26679 raft_consensus.cc:504] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f51376f2af254cbdaaf0a518f86631d7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41345 } }
I20260928 14:52:24.846146 26679 leader_election.cc:302] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f51376f2af254cbdaaf0a518f86631d7; no voters:
I20260928 14:52:24.846908 26679 leader_election.cc:288] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:52:24.847002 26684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:52:24.847604 26684 raft_consensus.cc:686] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [term 1 LEADER]: Becoming Leader. State: Replica: f51376f2af254cbdaaf0a518f86631d7, State: Running, Role: LEADER
I20260928 14:52:24.848109 26684 consensus_queue.cc:237] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f51376f2af254cbdaaf0a518f86631d7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41345 } }
I20260928 14:52:24.848906 26679 sys_catalog.cc:564] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:52:24.850870 26685 sys_catalog.cc:455] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f51376f2af254cbdaaf0a518f86631d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f51376f2af254cbdaaf0a518f86631d7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41345 } } }
I20260928 14:52:24.851264 26685 sys_catalog.cc:458] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [sys.catalog]: This master's current role is: LEADER
I20260928 14:52:24.850900 26686 sys_catalog.cc:455] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f51376f2af254cbdaaf0a518f86631d7. Latest consensus state: current_term: 1 leader_uuid: "f51376f2af254cbdaaf0a518f86631d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f51376f2af254cbdaaf0a518f86631d7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41345 } } }
I20260928 14:52:24.856223 26686 sys_catalog.cc:458] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7 [sys.catalog]: This master's current role is: LEADER
I20260928 14:52:24.856298 26690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:52:24.860265 26690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:52:24.861096 26679 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 14:52:24.877671 26690 catalog_manager.cc:1347] Generated new cluster ID: fde8eb06e25e4a8ab7930ee6e50b0d3a
I20260928 14:52:24.877818 26690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:52:24.898779 26690 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:52:24.900149 26690 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:52:24.907851 26690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7: Generated new TSK 0
I20260928 14:52:24.908790 26690 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:52:25.732951 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:57763/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41345
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:52:25 dist-test-slave-vk40 krb5kdc[26168](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790607145, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260928 14:52:26.206331 26724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:52:26.206789 26724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:52:26.206954 26724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:52:26.218544 26724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:52:26.218701 26724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:52:26.218917 26724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:52:26.234338 26724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.241.212:57763/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:52:26.235733 26724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:52:26.237138 26724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:52:26.255105 26734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:52:26.255537 26735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:52:26.257230 26724 server_base.cc:1034] running on GCE node
W20260928 14:52:26.260910 26737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:52:26.262108 26724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:52:26.263743 26724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:52:26.265754 26724 hybrid_clock.cc:648] HybridClock initialized: now 1790607146265642 us; error 78 us; skew 500 ppm
I20260928 14:52:26.266336 26724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:52:26 dist-test-slave-vk40 krb5kdc[26168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607146, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:52:26.276026 26724 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:52:26.278530 26724 webserver.cc:460] Webserver started at http://127.14.241.193:46139/ using document root <none> and password file <none>
I20260928 14:52:26.279323 26724 fs_manager.cc:359] Metadata directory not provided
I20260928 14:52:26.279505 26724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:52:26.279891 26724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:52:26 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607146, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:52:26.309407 26724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "cc6e10c62bb449dca635dd4caa4617f9"
format_stamp: "Formatted at 2026-09-28 14:52:26 on dist-test-slave-vk40"
server_key: "986ca9cd65a5acbebb833f6bd8fadbc5"
server_key_iv: "013ee63fecf9f796e651a72af0638aa5"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:26.310662 26724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "cc6e10c62bb449dca635dd4caa4617f9"
format_stamp: "Formatted at 2026-09-28 14:52:26 on dist-test-slave-vk40"
server_key: "986ca9cd65a5acbebb833f6bd8fadbc5"
server_key_iv: "013ee63fecf9f796e651a72af0638aa5"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:26.318162 26724 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.003s
I20260928 14:52:26.351040 26744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:52:26.353379 26724 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.010s sys 0.001s
I20260928 14:52:26.353710 26724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "cc6e10c62bb449dca635dd4caa4617f9"
format_stamp: "Formatted at 2026-09-28 14:52:26 on dist-test-slave-vk40"
server_key: "986ca9cd65a5acbebb833f6bd8fadbc5"
server_key_iv: "013ee63fecf9f796e651a72af0638aa5"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:26.354096 26724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:52:26.418114 26724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:52:26.423463 26724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:52:26.423882 26724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:52:26.431352 26724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:52:26.439677 26724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:52:26.439877 26724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:52:26.440234 26724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:52:26.442597 26724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
Sep 28 14:52:26 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607145, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:52:26.926963 26724 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:33793
I20260928 14:52:26.926954 26856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:33793 every 8 connection(s)
I20260928 14:52:26.928689 26724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:52:26.935875 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 26724
I20260928 14:52:26.936079 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:52:26.939133 15303 external_mini_cluster.cc:1428] Setting key b24683e74f8f869491a91541f2d0f1ef
Sep 28 14:52:26 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607146, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:52:26.961143 26860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:26.933172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:40263 (local address 127.14.241.254:41345)
0928 14:52:26.939050 (+ 5878us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:26.939058 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:26.940195 (+ 1137us) server_negotiation.cc:407] Connection header received
0928 14:52:26.943036 (+ 2841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:26.943042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:26.943120 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:26.943219 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:52:26.945477 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:26.946491 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:26.948877 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:26.949512 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:26.949647 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:26.954171 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:52:26.954178 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:52:26.954183 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:52:26.954221 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:52:26.957452 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:26.958401 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:26.958406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:26.958409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:26.958555 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:26.959078 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:26.959083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:26.959086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:26.959422 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:52:26.959556 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:26.960553 (+ 997us) server_negotiation.cc:299] Negotiation successful
0928 14:52:26.960758 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5737,"thread_start_us":5615,"threads_started":1}
I20260928 14:52:26.963614 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:26.932419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41345 (local address 127.14.241.193:40263)
0928 14:52:26.939619 (+ 7200us) negotiation.cc:107] Waiting for socket to connect
0928 14:52:26.939677 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 14:52:26.942595 (+ 2918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:52:26.943674 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:52:26.943686 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:52:26.944411 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:52:26.945245 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:26.945271 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:26.946697 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:26.946703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:26.948675 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:26.948691 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:26.949958 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:26.949966 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:26.950258 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:26.951120 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:52:26.951150 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:52:26.953914 (+ 2764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:52:26.957656 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:52:26.957665 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:52:26.957686 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:52:26.958257 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:52:26.958672 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:52:26.958678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:52:26.958682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:52:26.958940 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:52:26.959632 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:52:26.959640 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:52:26.960105 (+ 465us) client_negotiation.cc:769] Sending connection context
0928 14:52:26.961686 (+ 1581us) client_negotiation.cc:240] Negotiation successful
0928 14:52:26.962226 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6868,"thread_start_us":6733,"threads_started":1}
I20260928 14:52:26.965865 26857 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41345
I20260928 14:52:26.966203 26857 heartbeater.cc:464] Registering TS with master...
I20260928 14:52:26.967187 26857 heartbeater.cc:511] Master 127.14.241.254:41345 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
I20260928 14:52:26.970093 26644 ts_manager.cc:194] Registered new tserver with Master: cc6e10c62bb449dca635dd4caa4617f9 (127.14.241.193:33793)
I20260928 14:52:26.972110 26644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:40263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Sep 28 14:52:27 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607145, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:52:27.012109 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:57763/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41345
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:52:27.495092 26864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:52:27.495391 26864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:52:27.495553 26864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:52:27.502907 26864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:52:27.503038 26864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:52:27.503221 26864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:52:27.513576 26864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.241.212:57763/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:52:27.514992 26864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:52:27.516359 26864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:52:27.536810 26877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:52:27.536831 26878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:52:27.544430 26880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:52:27.544694 26864 server_base.cc:1034] running on GCE node
I20260928 14:52:27.545500 26864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:52:27.546552 26864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:52:27.547832 26864 hybrid_clock.cc:648] HybridClock initialized: now 1790607147547766 us; error 59 us; skew 500 ppm
I20260928 14:52:27.548189 26864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:52:27 dist-test-slave-vk40 krb5kdc[26168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607147, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:52:27.551298 26864 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:52:27.553948 26864 webserver.cc:460] Webserver started at http://127.14.241.194:45137/ using document root <none> and password file <none>
I20260928 14:52:27.554641 26864 fs_manager.cc:359] Metadata directory not provided
I20260928 14:52:27.554751 26864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:52:27.555048 26864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:52:27 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607147, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:52:27.584208 26864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "9e62d71c310e4c4baffb9ba91767d934"
format_stamp: "Formatted at 2026-09-28 14:52:27 on dist-test-slave-vk40"
server_key: "611a824cd9d9e877590f469b5ee0ed24"
server_key_iv: "8ff4e7f0f271eb90f5966c7ab7a26ccb"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:27.590592 26864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "9e62d71c310e4c4baffb9ba91767d934"
format_stamp: "Formatted at 2026-09-28 14:52:27 on dist-test-slave-vk40"
server_key: "611a824cd9d9e877590f469b5ee0ed24"
server_key_iv: "8ff4e7f0f271eb90f5966c7ab7a26ccb"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:27.611711 26864 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.012s sys 0.004s
I20260928 14:52:27.645001 26887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:52:27.647178 26864 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.007s sys 0.007s
I20260928 14:52:27.647373 26864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "9e62d71c310e4c4baffb9ba91767d934"
format_stamp: "Formatted at 2026-09-28 14:52:27 on dist-test-slave-vk40"
server_key: "611a824cd9d9e877590f469b5ee0ed24"
server_key_iv: "8ff4e7f0f271eb90f5966c7ab7a26ccb"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:27.647629 26864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:52:27.719627 26864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:52:27.725526 26864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:52:27.725885 26864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:52:27.727063 26864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:52:27.728850 26864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:52:27.728940 26864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:52:27.729063 26864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:52:27.729117 26864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:52:27.771359 26864 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:42125
I20260928 14:52:27.771473 26999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:42125 every 8 connection(s)
I20260928 14:52:27.772722 26864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:52:27.777233 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 26864
I20260928 14:52:27.777416 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:52:27.777958 15303 external_mini_cluster.cc:1428] Setting key 4b30a866f3f3c25d73256cb174cac70e
Sep 28 14:52:27 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607147, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
I20260928 14:52:27.807423 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:27.775619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:47749 (local address 127.14.241.254:41345)
0928 14:52:27.789619 (+ 14000us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:27.789625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:27.789645 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:52:27.789704 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:27.789709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:27.789792 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:27.789909 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:52:27.792376 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:27.793482 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:27.795787 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:27.796437 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:27.796558 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:27.800903 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:52:27.800911 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:52:27.800916 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:52:27.800963 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:52:27.804168 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:27.805101 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:27.805106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:27.805108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:27.805233 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:27.805654 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:27.805657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:27.805660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:27.805943 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:52:27.806075 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:27.806830 (+ 755us) server_negotiation.cc:299] Negotiation successful
0928 14:52:27.807061 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13864,"thread_start_us":13730,"threads_started":1}
I20260928 14:52:27.809170 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:27.776456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41345 (local address 127.14.241.194:47749)
0928 14:52:27.785598 (+ 9142us) negotiation.cc:107] Waiting for socket to connect
0928 14:52:27.785653 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 14:52:27.788398 (+ 2745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:52:27.790253 (+ 1855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:52:27.790266 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:52:27.791196 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:52:27.792152 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:27.792180 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:27.793668 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:27.793674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:27.795622 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:27.795638 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:27.796678 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:27.796686 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:27.796988 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:27.797775 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:52:27.797806 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:52:27.800639 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:52:27.804337 (+ 3698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:52:27.804345 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:52:27.804360 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:52:27.804965 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:52:27.805337 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:52:27.805343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:52:27.805347 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:52:27.805539 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:52:27.806074 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:52:27.806083 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:52:27.806438 (+ 355us) client_negotiation.cc:769] Sending connection context
0928 14:52:27.806691 (+ 253us) client_negotiation.cc:240] Negotiation successful
0928 14:52:27.807234 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8781,"thread_start_us":11779,"threads_started":1}
I20260928 14:52:27.811012 27000 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41345
I20260928 14:52:27.811318 27000 heartbeater.cc:464] Registering TS with master...
I20260928 14:52:27.812120 27000 heartbeater.cc:511] Master 127.14.241.254:41345 requested a full tablet report, sending...
I20260928 14:52:27.813889 26644 ts_manager.cc:194] Registered new tserver with Master: 9e62d71c310e4c4baffb9ba91767d934 (127.14.241.194:42125)
I20260928 14:52:27.815239 26644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:47749
I20260928 14:52:27.976329 26857 heartbeater.cc:503] Master 127.14.241.254:41345 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
W20260928 14:52:28.281137 15303 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.14.241.195: real 0.500s user 0.002s sys 0.015s
I20260928 14:52:28.281967 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:57763/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41345
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:52:28.632237 27006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:52:28.632539 27006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:52:28.632637 27006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:52:28.640079 27006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:52:28.640187 27006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:52:28.640326 27006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:52:28.650503 27006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.241.212:57763/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:52:28.651793 27006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:52:28.653156 27006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:52:28.666221 27012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:52:28.666388 27011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:52:28.666385 27014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:52:28.667263 27006 server_base.cc:1034] running on GCE node
I20260928 14:52:28.667979 27006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:52:28.669106 27006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:52:28.670329 27006 hybrid_clock.cc:648] HybridClock initialized: now 1790607148670270 us; error 49 us; skew 500 ppm
I20260928 14:52:28.670637 27006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:52:28 dist-test-slave-vk40 krb5kdc[26168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607148, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:52:28.673681 27006 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:52:28.675092 27006 webserver.cc:460] Webserver started at http://127.14.241.195:38621/ using document root <none> and password file <none>
I20260928 14:52:28.675727 27006 fs_manager.cc:359] Metadata directory not provided
I20260928 14:52:28.675827 27006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:52:28.676075 27006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:52:28 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607148, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:52:28.690668 27006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "576d5319315341ce989963dc897a73f0"
format_stamp: "Formatted at 2026-09-28 14:52:28 on dist-test-slave-vk40"
server_key: "6194f50ee395664371a2d7ad7941edfb"
server_key_iv: "f69dd64599b39b5a728035ac42a87b22"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:28.691495 27006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "576d5319315341ce989963dc897a73f0"
format_stamp: "Formatted at 2026-09-28 14:52:28 on dist-test-slave-vk40"
server_key: "6194f50ee395664371a2d7ad7941edfb"
server_key_iv: "f69dd64599b39b5a728035ac42a87b22"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:28.696777 27006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260928 14:52:28.711162 27021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:52:28.712934 27006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260928 14:52:28.713109 27006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "576d5319315341ce989963dc897a73f0"
format_stamp: "Formatted at 2026-09-28 14:52:28 on dist-test-slave-vk40"
server_key: "6194f50ee395664371a2d7ad7941edfb"
server_key_iv: "f69dd64599b39b5a728035ac42a87b22"
server_key_version: "kuduclusterkey@0"
I20260928 14:52:28.713364 27006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:52:28.727977 27006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:52:28.731843 27006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:52:28.732121 27006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:52:28.733122 27006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:52:28.734421 27006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:52:28.734489 27006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:52:28.734596 27006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:52:28.734644 27006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:52:28.768772 27006 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:44343
I20260928 14:52:28.768818 27133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:44343 every 8 connection(s)
I20260928 14:52:28.770006 27006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:52:28.773891 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 27006
I20260928 14:52:28.774072 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:52:28.774559 15303 external_mini_cluster.cc:1428] Setting key 4bbedf24c9bf4c695b88fd87536bc7d1
Sep 28 14:52:28 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607148, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:52:28.789915 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.772529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:49659 (local address 127.14.241.254:41345)
0928 14:52:28.773066 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:28.773072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:28.776537 (+ 3465us) server_negotiation.cc:407] Connection header received
0928 14:52:28.776599 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:28.776603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:28.776703 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:28.776813 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:52:28.778402 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.779163 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.780800 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.781232 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.781342 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.784603 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:52:28.784609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:52:28.784612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:52:28.784643 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:52:28.787070 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:28.787692 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:28.787695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:28.787698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:28.787811 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:28.788208 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:28.788213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:28.788216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:28.788566 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:52:28.788696 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:28.789399 (+ 703us) server_negotiation.cc:299] Negotiation successful
0928 14:52:28.789567 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":284,"threads_started":1}
I20260928 14:52:28.790850 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.773073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41345 (local address 127.14.241.195:49659)
0928 14:52:28.773874 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0928 14:52:28.773930 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 14:52:28.776162 (+ 2232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:52:28.777075 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:52:28.777086 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:52:28.777649 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:52:28.778258 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.778275 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.779345 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.779349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.780681 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.780693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.781407 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.781412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.781603 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.782187 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:52:28.782208 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:52:28.784401 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:52:28.787177 (+ 2776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:52:28.787183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:52:28.787194 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:52:28.787588 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:52:28.787904 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:52:28.787908 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:52:28.787911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:52:28.788115 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:52:28.788685 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:52:28.788693 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:52:28.789028 (+ 335us) client_negotiation.cc:769] Sending connection context
0928 14:52:28.789294 (+ 266us) client_negotiation.cc:240] Negotiation successful
0928 14:52:28.789811 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":308,"threads_started":1}
I20260928 14:52:28.792153 27134 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41345
I20260928 14:52:28.792457 27134 heartbeater.cc:464] Registering TS with master...
I20260928 14:52:28.793133 27134 heartbeater.cc:511] Master 127.14.241.254:41345 requested a full tablet report, sending...
I20260928 14:52:28.794476 26644 ts_manager.cc:194] Registered new tserver with Master: 576d5319315341ce989963dc897a73f0 (127.14.241.195:44343)
I20260928 14:52:28.795212 26644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:49659
I20260928 14:52:28.801960 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:52:28.819103 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.806135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52952 (local address 127.14.241.254:41345)
0928 14:52:28.806279 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:28.806290 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:28.806747 (+ 457us) server_negotiation.cc:407] Connection header received
0928 14:52:28.806896 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:28.806899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:28.806971 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:28.807042 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:52:28.808389 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.809700 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.810979 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.811646 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.811758 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.814009 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:52:28.814016 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:52:28.814021 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:52:28.814058 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:52:28.816867 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:28.817492 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:28.817496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:28.817498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:28.817610 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:28.817942 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:28.817945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:28.817947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:28.818211 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:52:28.818404 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:28.818666 (+ 262us) server_negotiation.cc:299] Negotiation successful
0928 14:52:28.818856 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 14:52:28.819140 27000 heartbeater.cc:503] Master 127.14.241.254:41345 was elected leader, sending a full tablet report...
I20260928 14:52:28.822890 26643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:52:28.829962 26643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:52:28.862825 27152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.842923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42416 (local address 127.14.241.195:44343)
0928 14:52:28.854590 (+ 11667us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:28.854599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:28.855210 (+ 611us) server_negotiation.cc:407] Connection header received
0928 14:52:28.855310 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:28.855317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:28.855531 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:28.855698 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:52:28.856649 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.858031 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.860381 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.861428 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.861548 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.861675 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:28.862316 (+ 641us) server_negotiation.cc:299] Negotiation successful
0928 14:52:28.862485 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11513,"thread_start_us":11398,"threads_started":1}
I20260928 14:52:28.863095 27148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.842455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:44343 (local address 127.0.0.1:42416)
0928 14:52:28.854955 (+ 12500us) negotiation.cc:107] Waiting for socket to connect
0928 14:52:28.854979 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 14:52:28.855093 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:52:28.855991 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:52:28.855995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:52:28.856025 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:52:28.856440 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.856452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.858190 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.858195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.860190 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.860206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.861840 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.861846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.862041 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.862092 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 14:52:28.862209 (+ 117us) client_negotiation.cc:240] Negotiation successful
0928 14:52:28.862394 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12392,"mutex_wait_us":120,"thread_start_us":11750,"threads_started":1}
I20260928 14:52:28.863345 27150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.842281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.14.241.194:42125)
0928 14:52:28.854984 (+ 12703us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:28.854991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:28.855017 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:52:28.855107 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:28.855114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:28.855343 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:28.855527 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:52:28.856357 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.857424 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.860402 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.861822 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.861996 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.862158 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:28.862889 (+ 731us) server_negotiation.cc:299] Negotiation successful
0928 14:52:28.863042 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12518,"thread_start_us":12379,"threads_started":1}
I20260928 14:52:28.863555 27151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.841863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:42125 (local address 127.0.0.1:39978)
0928 14:52:28.853969 (+ 12106us) negotiation.cc:107] Waiting for socket to connect
0928 14:52:28.854006 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:52:28.854468 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:52:28.855736 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:52:28.855740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:52:28.855764 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:52:28.856155 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.856172 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.857553 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.857558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.860201 (+ 2643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.860215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.862320 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.862327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.862482 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.862513 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 14:52:28.862595 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:52:28.862721 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11882,"mutex_wait_us":97,"thread_start_us":12994,"threads_started":1}
I20260928 14:52:28.864032 27147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.841725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33793 (local address 127.0.0.1:55342)
0928 14:52:28.853970 (+ 12245us) negotiation.cc:107] Waiting for socket to connect
0928 14:52:28.854006 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:52:28.854272 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:52:28.855108 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:52:28.855117 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:52:28.855147 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:52:28.855613 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.855632 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.857754 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.857759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.860586 (+ 2827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.860599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.863208 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.863214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.863401 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.863435 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 14:52:28.863529 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 14:52:28.863651 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11931,"mutex_wait_us":95,"spinlock_wait_cycles":135936,"thread_start_us":11772,"threads_started":1}
I20260928 14:52:28.864645 27149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.842047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55342 (local address 127.14.241.193:33793)
0928 14:52:28.853379 (+ 11332us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:28.853391 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:28.854476 (+ 1085us) server_negotiation.cc:407] Connection header received
0928 14:52:28.854589 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:28.854598 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:28.854873 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:28.855814 (+ 941us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:52:28.856045 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.857620 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.860747 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.862108 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.862921 (+ 813us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.863080 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:28.864225 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0928 14:52:28.864405 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11102,"thread_start_us":10922,"threads_started":1}
I20260928 14:52:28.867715 26792 tablet_service.cc:1467] Processing CreateTablet for tablet 7e48f8c326fc4c7f8c0271709612df93 (DEFAULT_TABLE table=test-table [id=2cccc4b079ef4b78b2b6f5e4063c36ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:52:28.867668 27069 tablet_service.cc:1467] Processing CreateTablet for tablet 7e48f8c326fc4c7f8c0271709612df93 (DEFAULT_TABLE table=test-table [id=2cccc4b079ef4b78b2b6f5e4063c36ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:52:28.867668 26935 tablet_service.cc:1467] Processing CreateTablet for tablet 7e48f8c326fc4c7f8c0271709612df93 (DEFAULT_TABLE table=test-table [id=2cccc4b079ef4b78b2b6f5e4063c36ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:52:28.869040 26792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e48f8c326fc4c7f8c0271709612df93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:52:28.869046 26935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e48f8c326fc4c7f8c0271709612df93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:52:28.869045 27069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e48f8c326fc4c7f8c0271709612df93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:52:28.880636 27153 tablet_bootstrap.cc:492] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: Bootstrap starting.
I20260928 14:52:28.883543 27154 tablet_bootstrap.cc:492] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0: Bootstrap starting.
I20260928 14:52:28.884092 27153 tablet_bootstrap.cc:654] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: Neither blocks nor log segments found. Creating new log.
I20260928 14:52:28.885408 27155 tablet_bootstrap.cc:492] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934: Bootstrap starting.
I20260928 14:52:28.885443 27153 log.cc:824] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: Log is configured to *not* fsync() on all Append() calls
I20260928 14:52:28.887482 27154 tablet_bootstrap.cc:654] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0: Neither blocks nor log segments found. Creating new log.
I20260928 14:52:28.888579 27155 tablet_bootstrap.cc:654] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934: Neither blocks nor log segments found. Creating new log.
I20260928 14:52:28.888612 27153 tablet_bootstrap.cc:492] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: No bootstrap required, opened a new log
I20260928 14:52:28.888787 27154 log.cc:824] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0: Log is configured to *not* fsync() on all Append() calls
I20260928 14:52:28.888873 27153 ts_tablet_manager.cc:1397] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260928 14:52:28.889856 27155 log.cc:824] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934: Log is configured to *not* fsync() on all Append() calls
I20260928 14:52:28.891789 27154 tablet_bootstrap.cc:492] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0: No bootstrap required, opened a new log
I20260928 14:52:28.892043 27154 ts_tablet_manager.cc:1397] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 14:52:28.892195 27155 tablet_bootstrap.cc:492] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934: No bootstrap required, opened a new log
I20260928 14:52:28.892460 27155 ts_tablet_manager.cc:1397] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.005s
I20260928 14:52:28.898881 27153 raft_consensus.cc:348] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6e10c62bb449dca635dd4caa4617f9" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33793 } } peers { permanent_uuid: "9e62d71c310e4c4baffb9ba91767d934" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42125 } } peers { permanent_uuid: "576d5319315341ce989963dc897a73f0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44343 } }
I20260928 14:52:28.899343 27153 raft_consensus.cc:374] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:52:28.899466 27153 raft_consensus.cc:729] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc6e10c62bb449dca635dd4caa4617f9, State: Initialized, Role: FOLLOWER
I20260928 14:52:28.899606 27155 raft_consensus.cc:348] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6e10c62bb449dca635dd4caa4617f9" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33793 } } peers { permanent_uuid: "9e62d71c310e4c4baffb9ba91767d934" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42125 } } peers { permanent_uuid: "576d5319315341ce989963dc897a73f0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44343 } }
I20260928 14:52:28.900013 27155 raft_consensus.cc:374] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:52:28.900112 27155 raft_consensus.cc:729] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e62d71c310e4c4baffb9ba91767d934, State: Initialized, Role: FOLLOWER
I20260928 14:52:28.900110 27153 consensus_queue.cc:260] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6e10c62bb449dca635dd4caa4617f9" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33793 } } peers { permanent_uuid: "9e62d71c310e4c4baffb9ba91767d934" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42125 } } peers { permanent_uuid: "576d5319315341ce989963dc897a73f0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44343 } }
I20260928 14:52:28.900651 27155 consensus_queue.cc:260] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6e10c62bb449dca635dd4caa4617f9" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33793 } } peers { permanent_uuid: "9e62d71c310e4c4baffb9ba91767d934" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42125 } } peers { permanent_uuid: "576d5319315341ce989963dc897a73f0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44343 } }
I20260928 14:52:28.902089 27153 ts_tablet_manager.cc:1428] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260928 14:52:28.903380 27154 raft_consensus.cc:348] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6e10c62bb449dca635dd4caa4617f9" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33793 } } peers { permanent_uuid: "9e62d71c310e4c4baffb9ba91767d934" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42125 } } peers { permanent_uuid: "576d5319315341ce989963dc897a73f0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44343 } }
I20260928 14:52:28.903682 27155 ts_tablet_manager.cc:1428] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260928 14:52:28.903844 27154 raft_consensus.cc:374] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:52:28.903986 27154 raft_consensus.cc:729] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 576d5319315341ce989963dc897a73f0, State: Initialized, Role: FOLLOWER
I20260928 14:52:28.904886 27154 consensus_queue.cc:260] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6e10c62bb449dca635dd4caa4617f9" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33793 } } peers { permanent_uuid: "9e62d71c310e4c4baffb9ba91767d934" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42125 } } peers { permanent_uuid: "576d5319315341ce989963dc897a73f0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44343 } }
I20260928 14:52:28.908875 27160 raft_consensus.cc:482] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:52:28.909159 27160 raft_consensus.cc:504] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6e10c62bb449dca635dd4caa4617f9" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33793 } } peers { permanent_uuid: "9e62d71c310e4c4baffb9ba91767d934" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42125 } } peers { permanent_uuid: "576d5319315341ce989963dc897a73f0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44343 } }
I20260928 14:52:28.911105 27160 leader_election.cc:288] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc6e10c62bb449dca635dd4caa4617f9 (127.14.241.193:33793), 576d5319315341ce989963dc897a73f0 (127.14.241.195:44343)
I20260928 14:52:28.913173 27154 ts_tablet_manager.cc:1428] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0: Time spent starting tablet: real 0.021s user 0.007s sys 0.013s
I20260928 14:52:28.913659 27134 heartbeater.cc:503] Master 127.14.241.254:41345 was elected leader, sending a full tablet report...
I20260928 14:52:28.918653 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.911471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33793 (local address 127.14.241.194:34547)
0928 14:52:28.912738 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0928 14:52:28.912751 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:52:28.912889 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:52:28.913189 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:52:28.913193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:52:28.913216 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:52:28.913584 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.913595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.914873 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.914879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.916454 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.916469 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.918029 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.918037 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.918172 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.918196 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 14:52:28.918268 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 14:52:28.918358 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1170,"thread_start_us":1012,"threads_started":1}
I20260928 14:52:28.918712 27152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.911733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:35189 (local address 127.14.241.195:44343)
0928 14:52:28.911892 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:28.911898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:28.912235 (+ 337us) server_negotiation.cc:407] Connection header received
0928 14:52:28.912463 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:28.912468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:28.912538 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:28.912596 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:52:28.913387 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.914276 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.916474 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.917693 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.917840 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.917912 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:28.918395 (+ 483us) server_negotiation.cc:299] Negotiation successful
0928 14:52:28.918510 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 14:52:28.918694 27149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.911897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:34547 (local address 127.14.241.193:33793)
0928 14:52:28.912071 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:28.912075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:28.912855 (+ 780us) server_negotiation.cc:407] Connection header received
0928 14:52:28.913026 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:28.913031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:28.913089 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:28.913154 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:52:28.913750 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.914745 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.916592 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.917818 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.917955 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.918020 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:28.918319 (+ 299us) server_negotiation.cc:299] Negotiation successful
0928 14:52:28.918447 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 14:52:28.919387 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.911285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:44343 (local address 127.14.241.194:35189)
0928 14:52:28.912117 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0928 14:52:28.912139 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 14:52:28.912275 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:52:28.912821 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:52:28.912825 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:52:28.912852 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:52:28.913198 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.913211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.914405 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.914411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.916265 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:52:28.916316 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.918073 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.918080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:52:28.918225 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.918248 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 14:52:28.919018 (+ 770us) client_negotiation.cc:240] Negotiation successful
0928 14:52:28.919110 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":564,"threads_started":1}
I20260928 14:52:28.919795 26812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e48f8c326fc4c7f8c0271709612df93" candidate_uuid: "9e62d71c310e4c4baffb9ba91767d934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc6e10c62bb449dca635dd4caa4617f9" is_pre_election: true
I20260928 14:52:28.920360 26812 raft_consensus.cc:2454] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e62d71c310e4c4baffb9ba91767d934 in term 0.
I20260928 14:52:28.920998 26891 leader_election.cc:302] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e62d71c310e4c4baffb9ba91767d934, cc6e10c62bb449dca635dd4caa4617f9; no voters:
I20260928 14:52:28.921404 27160 raft_consensus.cc:2781] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:52:28.921552 27160 raft_consensus.cc:482] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:52:28.921661 27160 raft_consensus.cc:3037] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:52:28.921587 27089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e48f8c326fc4c7f8c0271709612df93" candidate_uuid: "9e62d71c310e4c4baffb9ba91767d934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "576d5319315341ce989963dc897a73f0" is_pre_election: true
I20260928 14:52:28.922034 27089 raft_consensus.cc:2454] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e62d71c310e4c4baffb9ba91767d934 in term 0.
I20260928 14:52:28.923928 27160 raft_consensus.cc:504] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6e10c62bb449dca635dd4caa4617f9" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33793 } } peers { permanent_uuid: "9e62d71c310e4c4baffb9ba91767d934" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42125 } } peers { permanent_uuid: "576d5319315341ce989963dc897a73f0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44343 } }
I20260928 14:52:28.924517 27160 leader_election.cc:288] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [CANDIDATE]: Term 1 election: Requested vote from peers cc6e10c62bb449dca635dd4caa4617f9 (127.14.241.193:33793), 576d5319315341ce989963dc897a73f0 (127.14.241.195:44343)
I20260928 14:52:28.924917 27089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e48f8c326fc4c7f8c0271709612df93" candidate_uuid: "9e62d71c310e4c4baffb9ba91767d934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "576d5319315341ce989963dc897a73f0"
I20260928 14:52:28.924924 26812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e48f8c326fc4c7f8c0271709612df93" candidate_uuid: "9e62d71c310e4c4baffb9ba91767d934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc6e10c62bb449dca635dd4caa4617f9"
I20260928 14:52:28.925078 27089 raft_consensus.cc:3037] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:52:28.925093 26812 raft_consensus.cc:3037] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:52:28.927670 26812 raft_consensus.cc:2454] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e62d71c310e4c4baffb9ba91767d934 in term 1.
I20260928 14:52:28.928035 26891 leader_election.cc:302] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e62d71c310e4c4baffb9ba91767d934, cc6e10c62bb449dca635dd4caa4617f9; no voters:
I20260928 14:52:28.928061 27089 raft_consensus.cc:2454] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e62d71c310e4c4baffb9ba91767d934 in term 1.
I20260928 14:52:28.928344 27160 raft_consensus.cc:2781] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:52:28.928844 27160 raft_consensus.cc:686] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 1 LEADER]: Becoming Leader. State: Replica: 9e62d71c310e4c4baffb9ba91767d934, State: Running, Role: LEADER
I20260928 14:52:28.929358 27160 consensus_queue.cc:237] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6e10c62bb449dca635dd4caa4617f9" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33793 } } peers { permanent_uuid: "9e62d71c310e4c4baffb9ba91767d934" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42125 } } peers { permanent_uuid: "576d5319315341ce989963dc897a73f0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44343 } }
I20260928 14:52:28.933593 26643 catalog_manager.cc:5495] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e62d71c310e4c4baffb9ba91767d934 (127.14.241.194). New cstate: current_term: 1 leader_uuid: "9e62d71c310e4c4baffb9ba91767d934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6e10c62bb449dca635dd4caa4617f9" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e62d71c310e4c4baffb9ba91767d934" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 42125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "576d5319315341ce989963dc897a73f0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44343 } health_report { overall_health: UNKNOWN } } }
W20260928 14:52:28.937935 26858 tablet.cc:2369] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:52:28.967690 27150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:28.962590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39986 (local address 127.14.241.194:42125)
0928 14:52:28.962785 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:28.962790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:28.962847 (+ 57us) server_negotiation.cc:407] Connection header received
0928 14:52:28.963009 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:28.963013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:28.963086 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:28.963198 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:52:28.963509 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.964258 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.965683 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:28.966075 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:28.966157 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:28.966696 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:52:28.967200 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:52:28.967320 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:28.967391 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 14:52:28.967454 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:52:28.975371 27089 raft_consensus.cc:1261] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0 [term 1 FOLLOWER]: Refusing update from remote peer 9e62d71c310e4c4baffb9ba91767d934: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:52:28.975361 26812 raft_consensus.cc:1261] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9 [term 1 FOLLOWER]: Refusing update from remote peer 9e62d71c310e4c4baffb9ba91767d934: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:52:28.976224 27164 consensus_queue.cc:1035] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "576d5319315341ce989963dc897a73f0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:52:28.976575 27160 consensus_queue.cc:1035] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc6e10c62bb449dca635dd4caa4617f9" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:52:29.001304 26644 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:52:29.001633 26644 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:52:29.005229 26644 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f51376f2af254cbdaaf0a518f86631d7: Sending DeleteTablet for 3 replicas of tablet 7e48f8c326fc4c7f8c0271709612df93
I20260928 14:52:29.006366 26935 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e48f8c326fc4c7f8c0271709612df93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:52:29 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:39978
I20260928 14:52:29.006369 26792 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e48f8c326fc4c7f8c0271709612df93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:52:29 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:55342
I20260928 14:52:29.006623 27069 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e48f8c326fc4c7f8c0271709612df93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:52:29 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:42416
I20260928 14:52:29.008787 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 26724
I20260928 14:52:29.021839 27179 tablet_replica.cc:330] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0: stopping tablet replica
I20260928 14:52:29.022562 27179 raft_consensus.cc:2229] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:52:29.023227 27179 raft_consensus.cc:2258] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:52:29.025971 27179 ts_tablet_manager.cc:1905] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:52:29.033313 27179 ts_tablet_manager.cc:1918] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:52:29.033581 27179 log.cc:1196] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/7e48f8c326fc4c7f8c0271709612df93
I20260928 14:52:29.034191 27179 ts_tablet_manager.cc:1939] T 7e48f8c326fc4c7f8c0271709612df93 P 576d5319315341ce989963dc897a73f0: Deleting consensus metadata
I20260928 14:52:29.037395 26629 catalog_manager.cc:4841] TS 576d5319315341ce989963dc897a73f0 (127.14.241.195:44343): tablet 7e48f8c326fc4c7f8c0271709612df93 (table test-table [id=2cccc4b079ef4b78b2b6f5e4063c36ff]) successfully deleted
I20260928 14:52:29.039284 27178 tablet_replica.cc:330] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: stopping tablet replica
I20260928 14:52:29.039909 27178 raft_consensus.cc:2229] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:52:29.040665 27178 raft_consensus.cc:2258] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:52:29.041273 27177 tablet_replica.cc:330] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934: stopping tablet replica
I20260928 14:52:29.041641 27177 raft_consensus.cc:2229] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:52:29.042256 27177 raft_consensus.cc:2258] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:52:29.044739 27178 ts_tablet_manager.cc:1905] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:52:29.046118 27177 ts_tablet_manager.cc:1905] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 14:52:29 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:52:29.052238 27178 ts_tablet_manager.cc:1918] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:52:29.052616 27178 log.cc:1196] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/7e48f8c326fc4c7f8c0271709612df93
I20260928 14:52:29.053201 27178 ts_tablet_manager.cc:1939] T 7e48f8c326fc4c7f8c0271709612df93 P cc6e10c62bb449dca635dd4caa4617f9: Deleting consensus metadata
I20260928 14:52:29.054590 27177 ts_tablet_manager.cc:1918] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:52:29.054840 27177 log.cc:1196] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/7e48f8c326fc4c7f8c0271709612df93
I20260928 14:52:29.055285 26631 catalog_manager.cc:4841] TS cc6e10c62bb449dca635dd4caa4617f9 (127.14.241.193:33793): tablet 7e48f8c326fc4c7f8c0271709612df93 (table test-table [id=2cccc4b079ef4b78b2b6f5e4063c36ff]) successfully deleted
I20260928 14:52:29.055423 27177 ts_tablet_manager.cc:1939] T 7e48f8c326fc4c7f8c0271709612df93 P 9e62d71c310e4c4baffb9ba91767d934: Deleting consensus metadata
I20260928 14:52:29.055974 27149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:29.009892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.14.241.193:33793)
0928 14:52:29.010079 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:29.010086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:29.033922 (+ 23836us) server_negotiation.cc:407] Connection header received
0928 14:52:29.034495 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:29.034503 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:29.034600 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:29.034692 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:52:29.038715 (+ 4023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:29.040000 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:29.042101 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:29.042720 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:29.042854 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:29.048333 (+ 5479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:52:29.048349 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:52:29.048367 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:52:29.048423 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:52:29.052391 (+ 3968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:29.053431 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:29.053440 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:29.053444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:29.053639 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:29.054122 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:29.054126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:29.054130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:29.054598 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:52:29.054902 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:29.055475 (+ 573us) server_negotiation.cc:299] Negotiation successful
0928 14:52:29.055692 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:52:29.057156 26628 catalog_manager.cc:4841] TS 9e62d71c310e4c4baffb9ba91767d934 (127.14.241.194:42125): tablet 7e48f8c326fc4c7f8c0271709612df93 (table test-table [id=2cccc4b079ef4b78b2b6f5e4063c36ff]) successfully deleted
I20260928 14:52:29.057406 26852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:52:29.179032 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 26724
I20260928 14:52:29.193127 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 26864
Sep 28 14:52:29 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:52:29.210765 27150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:29.193895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39992 (local address 127.14.241.194:42125)
0928 14:52:29.194097 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:29.194102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:29.194125 (+ 23us) server_negotiation.cc:407] Connection header received
0928 14:52:29.194261 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:29.194267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:29.194353 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:29.194441 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:52:29.195672 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:29.196561 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:29.198111 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:29.198705 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:29.198811 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:29.204372 (+ 5561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:52:29.204388 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:52:29.204404 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:52:29.204449 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:52:29.207575 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:29.208534 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:29.208544 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:29.208548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:29.208776 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:29.209186 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:29.209189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:29.209191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:29.209565 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:52:29.209828 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:29.210145 (+ 317us) server_negotiation.cc:299] Negotiation successful
0928 14:52:29.210385 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 14:52:29.211447 26995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:52:29.325153 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 26864
I20260928 14:52:29.339819 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 27006
Sep 28 14:52:29 dist-test-slave-vk40 krb5kdc[26168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:52:29.354840 27152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:52:29.340513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42418 (local address 127.14.241.195:44343)
0928 14:52:29.340683 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 14:52:29.340688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:52:29.340714 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:52:29.340896 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:52:29.340901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:52:29.340978 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:52:29.341065 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:52:29.342327 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:29.343072 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:29.344348 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:52:29.344745 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:52:29.344852 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:52:29.349313 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:52:29.349329 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:52:29.349345 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:52:29.349382 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:52:29.351946 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:29.352715 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:29.352723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:29.352725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:29.352848 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:52:29.353254 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:52:29.353260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:52:29.353264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:52:29.353657 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:52:29.353918 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0928 14:52:29.354285 (+ 367us) server_negotiation.cc:299] Negotiation successful
0928 14:52:29.354517 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 14:52:29.355454 27129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:52:29.443848 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 27006
I20260928 14:52:29.457072 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 26612
I20260928 14:52:29.458122 26674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:52:29.569619 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 26612
I20260928 14:52:29.590744 15303 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 14:52:29.925685 15303 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 14:52:29.925911 15303 mini_ranger.cc:66] Stopping Ranger...
I20260928 14:52:30.261552 15303 mini_ranger.cc:68] Stopped Ranger
2026-09-28 14:52:30.261 UTC [26198] LOG: received smart shutdown request
2026-09-28 14:52:30.265 UTC [26198] LOG: background worker "logical replication launcher" (PID 26207) exited with exit code 1
2026-09-28 14:52:30.265 UTC [26202] LOG: shutting down
2026-09-28 14:52:30.397 UTC [26198] LOG: database system is shut down
2026-09-28T14:52:30Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50801 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:52:30 dist-test-slave-vk40 krb5kdc[27193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:52:30 dist-test-slave-vk40 krb5kdc[27193](info): set up 2 sockets
Sep 28 14:52:30 dist-test-slave-vk40 krb5kdc[27193](info): commencing operation
krb5kdc: starting...
W20260928 14:52:31.547425 15303 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:52:31 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:52:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:52:31Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/rangeradmin_127.14.241.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/rangeradmin_127.14.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/rangerlookup_127.14.241.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/rangerlookup_127.14.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/HTTP_127.14.241.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/HTTP_127.14.241.212@KRBTEST.COM.keytab.
I20260928 14:52:31.863267 15303 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKMueFh/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/postgres -l logfile start
2026-09-28 14:52:34.306 UTC [27223] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 14:52:34.306 UTC [27223] LOG: listening on IPv4 address "127.14.241.212", port 35175
2026-09-28 14:52:34.309 UTC [27223] LOG: listening on Unix socket "/tmp/.s.PGSQL.35175"
2026-09-28 14:52:34.329 UTC [27226] LOG: database system was shut down at 2026-09-28 14:52:32 UTC
2026-09-28 14:52:34.333 UTC [27223] LOG: database system is ready to accept connections
I20260928 14:52:35.222810 15303 mini_postgres.cc:96] Postgres bound to 35175
2026-09-28 14:52:35.238 UTC [27234] FATAL: database "slave" does not exist
127.14.241.212:35175 - accepting connections
I20260928 14:52:35.239513 15303 mini_ranger.cc:161] Starting Ranger...
I20260928 14:52:35.263813 15303 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 14:52:35.741056 15303 mini_ranger.cc:87] Created ranger Postgres database
I20260928 14:52:35.741168 15303 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-admin
2026-09-28 14:52:36,018 [I] DB FLAVOR :POSTGRES
2026-09-28 14:52:36,019 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:52:36,019 [I] Checking connection..
2026-09-28 14:52:36,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 14:52:36,277 [I] Checking connection passed.
2026-09-28 14:52:36,278 [I] --------- Verifying version history table ---------
2026-09-28 14:52:36,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:52:36,523 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:52:36,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:52:36,754 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:52:36,754 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 14:52:36,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 14:52:36.997 UTC [27298] WARNING: there is no transaction in progress
2026-09-28 14:52:37,002 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 14:52:37,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:52:37,241 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 14:52:37,242 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:52:37,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 14:52:37,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 14:52:37,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vk40.c.gcp-upstream.internal','N') ;"
2026-09-28 14:52:37,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:52:38,224 [I] Table x_portal_user does not exist in database ranger
2026-09-28 14:52:38,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:52:38,477 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 14:52:38,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:52:38,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:52:38,988 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 14:52:38,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 14:52:39.956 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:39.972 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:39.986 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.001 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.021 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.041 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.107 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.114 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.128 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.137 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.147 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.161 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.170 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.176 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.183 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40.189 UTC [27406] WARNING: there is no transaction in progress
2026-09-28 14:52:40,835 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 14:52:40,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 14:52:41,069 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 14:52:41,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:52:41,343 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 14:52:41,344 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:52:41,594 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 14:52:41,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:52:41,841 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:52:42,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vk40.c.gcp-upstream.internal';"
2026-09-28 14:52:42,331 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 14:52:42,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 14:52:42,569 [I] DB_PATCHES have already been applied
I20260928 14:52:42.574867 15303 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKMueFh/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 14:52:42.574956 15303 mini_ranger.cc:193] Using host: 127.14.241.212
I20260928 14:52:42.588779 15303 mini_ranger.cc:227] Ranger admin URL: http://127.14.241.212:44017
Sep 28, 2026 2:52:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:52:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:52:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 2:52:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 2:52:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 2:52:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 2:52:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 14:52:43.632078 15303 mini_ranger.cc:160] Time spent starting Ranger: real 8.393s user 0.000s sys 0.039s
Sep 28, 2026 2:53:02 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 2:53:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 28, 2026 2:53:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 2:53:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 2:53:03 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 14:53:04.369103 15303 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/rangerkms_127.14.241.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/rangerkms_127.14.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.241.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.241.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.241.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/HTTP_127.14.241.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.241.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/HTTP_127.14.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 14:53:04 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607184, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 14:53:04.556313 15303 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 14:53:04.576874 15303 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 14:53:05.482887 15303 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 14:53:05.482982 15303 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-kms
I20260928 14:53:05.483017 15303 mini_ranger_kms.cc:227] Using postgres at 127.14.241.212:35175
2026-09-28 14:53:06,057 [I] DB FLAVOR :POSTGRES
2026-09-28 14:53:06,057 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:53:06,057 [I] Checking connection
2026-09-28 14:53:06,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 14:53:06,300 [I] connection success
2026-09-28 14:53:06,301 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 14:53:06,301 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 14:53:06,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 14:53:06,539 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 14:53:06,539 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:53:06,539 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 14:53:06,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.241.212:35175/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 14:53:06,818 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 14:53:07.296660 15303 mini_ranger_kms.cc:310] Created kms service
I20260928 14:53:07.487283 15303 mini_ranger_kms.cc:326] Created kudu user
I20260928 14:53:07.532752 15303 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 14:53:07 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607184, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:53:07.944649 15303 mini_ranger_kms.cc:384] Added ranger policy
I20260928 14:53:07.944952 15303 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKMueFh/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKMueFh/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 14:53:07.944993 15303 mini_ranger_kms.cc:240] Using host: 127.14.241.212
I20260928 14:53:07.963027 15303 mini_ranger_kms.cc:276] Ranger KMS PID: 27586
I20260928 14:53:07.963152 15303 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.241.212:47345
Sep 28, 2026 2:53:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:53:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:53:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 2:53:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 2:53:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 2:53:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 14:53:10.039636 15303 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.483s user 0.088s sys 0.284s
I20260928 14:53:10.039901 15303 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 14:53:10.040059 15303 mini_ranger_kms.cc:401] 127.14.241.212:47345/kms/v1/keys
Sep 28 14:53:10 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790607190, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.241.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 14:53:10 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607190, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.241.212@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
Sep 28 14:53:10 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607190, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.241.212@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
Sep 28 14:53:13 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:53:13.180743 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46527
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:46527
--ranger_config_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:47345/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:13.505962 27637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:13.506271 27637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:13.506390 27637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:13.513736 27637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:53:13.513854 27637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:13.513909 27637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:13.513988 27637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:53:13.514043 27637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:53:13.524506 27637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.241.212:47345/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:46527
--ranger_config_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:13.525831 27637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:13.527256 27637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:13.538930 27642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:13.539005 27645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:13.538967 27643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:13.539950 27637 server_base.cc:1034] running on GCE node
I20260928 14:53:13.540781 27637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:13.542215 27637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:13.543421 27637 hybrid_clock.cc:648] HybridClock initialized: now 1790607193543377 us; error 41 us; skew 500 ppm
I20260928 14:53:13.543766 27637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:13 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607193, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:13.546877 27637 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:53:13.548447 27637 webserver.cc:460] Webserver started at http://127.14.241.254:45559/ using document root <none> and password file <none>
I20260928 14:53:13.549101 27637 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:13.549212 27637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:13.549510 27637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:53:13 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607193, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:53:13.608762 27637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4"
format_stamp: "Formatted at 2026-09-28 14:53:13 on dist-test-slave-vk40"
server_key: "c58668c674ab79ce58e3e2b4b60941be"
server_key_iv: "1c3a96e81b752c0c6467e63e2c1e7f13"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:13.609916 27637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4"
format_stamp: "Formatted at 2026-09-28 14:53:13 on dist-test-slave-vk40"
server_key: "c58668c674ab79ce58e3e2b4b60941be"
server_key_iv: "1c3a96e81b752c0c6467e63e2c1e7f13"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:13.617208 27637 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260928 14:53:13.638672 27652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:13.640204 27637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:53:13.640431 27637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4"
format_stamp: "Formatted at 2026-09-28 14:53:13 on dist-test-slave-vk40"
server_key: "c58668c674ab79ce58e3e2b4b60941be"
server_key_iv: "1c3a96e81b752c0c6467e63e2c1e7f13"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:13.640642 27637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:13.669124 27637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:13.672976 27637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:13.673316 27637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:13.690866 27637 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:46527
I20260928 14:53:13.690887 27703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:46527 every 8 connection(s)
I20260928 14:53:13.692201 27637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:53:13.694969 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 27637
I20260928 14:53:13.695182 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:53:13.696026 15303 external_mini_cluster.cc:1428] Setting key efac42ec5e8153e472c9c89e9c236b94
I20260928 14:53:13.696472 27704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:13.710919 27704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4: Bootstrap starting.
Sep 28 14:53:13 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:13.714208 27704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:13.715269 27704 log.cc:824] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:13.718146 27704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4: No bootstrap required, opened a new log
I20260928 14:53:13.719189 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:13.698129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51764 (local address 127.14.241.254:46527)
0928 14:53:13.698882 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:13.698893 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:13.698941 (+ 48us) server_negotiation.cc:407] Connection header received
0928 14:53:13.701072 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:13.701097 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:13.701542 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:13.701913 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:13.703538 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:13.704947 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:13.706789 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:13.707545 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:13.707696 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:13.712082 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:13.712091 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:13.712107 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:13.712157 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:13.715078 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:13.715814 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:13.715821 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:13.715828 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:13.716017 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:13.716410 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:13.716414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:13.716418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:13.716967 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:13.717256 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:13.717698 (+ 442us) server_negotiation.cc:299] Negotiation successful
0928 14:53:13.718096 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":359,"threads_started":1}
I20260928 14:53:13.725690 27704 raft_consensus.cc:348] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } }
I20260928 14:53:13.726058 27704 raft_consensus.cc:374] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:13.726150 27704 raft_consensus.cc:729] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec18ba7ae5b45eeb4040b746c3bd6e4, State: Initialized, Role: FOLLOWER
I20260928 14:53:13.726694 27704 consensus_queue.cc:260] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } }
I20260928 14:53:13.726953 27704 raft_consensus.cc:388] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:53:13.727089 27704 raft_consensus.cc:482] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:53:13.727248 27704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:13.729367 27704 raft_consensus.cc:504] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } }
I20260928 14:53:13.729844 27704 leader_election.cc:302] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cec18ba7ae5b45eeb4040b746c3bd6e4; no voters:
I20260928 14:53:13.730381 27704 leader_election.cc:288] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:53:13.730500 27709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:53:13.731112 27709 raft_consensus.cc:686] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 1 LEADER]: Becoming Leader. State: Replica: cec18ba7ae5b45eeb4040b746c3bd6e4, State: Running, Role: LEADER
I20260928 14:53:13.731604 27709 consensus_queue.cc:237] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } }
I20260928 14:53:13.732345 27704 sys_catalog.cc:564] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:53:13.734227 27710 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } } }
I20260928 14:53:13.734596 27710 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:13.736176 27714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:53:13.736135 27711 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec18ba7ae5b45eeb4040b746c3bd6e4. Latest consensus state: current_term: 1 leader_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } } }
I20260928 14:53:13.736553 27711 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:13.741434 27714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:53:13.746651 27704 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 14:53:13.758888 27714 catalog_manager.cc:1347] Generated new cluster ID: 69a202bbcdab47e3b0ad08ef299c1f73
I20260928 14:53:13.759055 27714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:53:13.780596 27714 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:53:13.782635 27714 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:53:13.791872 27714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4: Generated new TSK 0
I20260928 14:53:13.792980 27714 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:53:14.513090 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:47345/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46527
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:53:14 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790607194, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260928 14:53:14.950753 27749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:14.951244 27749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:14.951459 27749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:14.962999 27749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:14.963204 27749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:14.963426 27749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:53:14.979045 27749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.241.212:47345/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:14.981019 27749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:14.982870 27749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:14.999477 27760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:14.999902 27759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:15.003703 27762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:15.004419 27749 server_base.cc:1034] running on GCE node
I20260928 14:53:15.005180 27749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:15.006477 27749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:15.007792 27749 hybrid_clock.cc:648] HybridClock initialized: now 1790607195007725 us; error 69 us; skew 500 ppm
I20260928 14:53:15.008189 27749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:15 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607195, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:15.011638 27749 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:53:15.013576 27749 webserver.cc:460] Webserver started at http://127.14.241.193:39173/ using document root <none> and password file <none>
I20260928 14:53:15.014581 27749 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:15.014752 27749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:15.015175 27749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:53:15 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607195, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:53:15.032846 27749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "782e83676db4470d9e453c99f846df55"
format_stamp: "Formatted at 2026-09-28 14:53:15 on dist-test-slave-vk40"
server_key: "bb3aed8fa43b3b30401cab47d106f37f"
server_key_iv: "10aab69eaf1890c994e72caab6cda018"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:15.034083 27749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "782e83676db4470d9e453c99f846df55"
format_stamp: "Formatted at 2026-09-28 14:53:15 on dist-test-slave-vk40"
server_key: "bb3aed8fa43b3b30401cab47d106f37f"
server_key_iv: "10aab69eaf1890c994e72caab6cda018"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:15.060346 27749 fs_manager.cc:691] Time spent creating directory manager: real 0.026s user 0.008s sys 0.015s
I20260928 14:53:15.079159 27769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:15.080703 27749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 14:53:15.080917 27749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "782e83676db4470d9e453c99f846df55"
format_stamp: "Formatted at 2026-09-28 14:53:15 on dist-test-slave-vk40"
server_key: "bb3aed8fa43b3b30401cab47d106f37f"
server_key_iv: "10aab69eaf1890c994e72caab6cda018"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:15.081141 27749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:15.145681 27749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:15.158203 27749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:15.160625 27749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:15.166577 27749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:15.180166 27749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:15.180388 27749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260928 14:53:15.180576 27749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:15.180678 27749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:53:15 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607194, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
Sep 28 14:53:15 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607194, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:53:15.860517 27749 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:36201
I20260928 14:53:15.860548 27883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:36201 every 8 connection(s)
I20260928 14:53:15.862114 27749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:53:15.867525 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 27749
I20260928 14:53:15.867712 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:53:15.868503 15303 external_mini_cluster.cc:1428] Setting key 9110c7a58e11111a6a36816dfb2cd955
Sep 28 14:53:15 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607195, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:15.892733 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:15.865476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:58841 (local address 127.14.241.254:46527)
0928 14:53:15.866026 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:15.866033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:15.866924 (+ 891us) server_negotiation.cc:407] Connection header received
0928 14:53:15.870542 (+ 3618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:15.870547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:15.870625 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:15.870703 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:15.873302 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:15.874562 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:15.877356 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:15.878077 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:15.878183 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:15.883530 (+ 5347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:15.883538 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:15.883544 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:15.883594 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:15.887497 (+ 3903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:15.889433 (+ 1936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:15.889441 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:15.889445 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:15.889625 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:15.890273 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:15.890279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:15.890283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:15.890681 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:15.890833 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:15.891989 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0928 14:53:15.892257 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":249,"threads_started":1}
I20260928 14:53:15.895321 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:15.865947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46527 (local address 127.14.241.193:58841)
0928 14:53:15.866749 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:15.866812 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:15.870208 (+ 3396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:15.871348 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:15.871364 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:15.872090 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:15.873074 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:15.873102 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:15.874793 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:15.874811 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:15.877127 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:15.877149 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:15.878618 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:15.878625 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:15.878995 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:15.879968 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:15.880007 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:15.883171 (+ 3164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:15.888513 (+ 5342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:15.888525 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:15.888548 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:15.889259 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:15.889778 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:15.889783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:15.889787 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:15.890132 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:15.890937 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:15.890947 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:15.891493 (+ 546us) client_negotiation.cc:769] Sending connection context
0928 14:53:15.893020 (+ 1527us) client_negotiation.cc:240] Negotiation successful
0928 14:53:15.893714 (+ 694us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":307,"threads_started":1}
I20260928 14:53:15.898838 27884 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46527
I20260928 14:53:15.899204 27884 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:15.900060 27884 heartbeater.cc:511] Master 127.14.241.254:46527 requested a full tablet report, sending...
I20260928 14:53:15.902401 27669 ts_manager.cc:194] Registered new tserver with Master: 782e83676db4470d9e453c99f846df55 (127.14.241.193:36201)
I20260928 14:53:15.903914 27669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:58841
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:53:15.958092 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:47345/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46527
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:16.315085 27890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:16.315402 27890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:16.315526 27890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:16.322962 27890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:16.323101 27890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:16.323256 27890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:53:16.333474 27890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.241.212:47345/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:16.334784 27890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:16.336076 27890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:16.348608 27902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:16.348649 27903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:16.348773 27905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:16.349265 27890 server_base.cc:1034] running on GCE node
I20260928 14:53:16.349817 27890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:16.350750 27890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:16.351910 27890 hybrid_clock.cc:648] HybridClock initialized: now 1790607196351856 us; error 58 us; skew 500 ppm
I20260928 14:53:16.352238 27890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:16 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607196, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:16.360668 27890 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:53:16.362192 27890 webserver.cc:460] Webserver started at http://127.14.241.194:42127/ using document root <none> and password file <none>
I20260928 14:53:16.362884 27890 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:16.363000 27890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:16.363291 27890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:53:16 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607196, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:53:16.378376 27890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "3bc4c56232214648a0b28b627c44ca24"
format_stamp: "Formatted at 2026-09-28 14:53:16 on dist-test-slave-vk40"
server_key: "a267e2293565d32a8f12144b3882e04d"
server_key_iv: "2774a670f4e0ba88674bfb7d6ce810e6"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:16.379205 27890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "3bc4c56232214648a0b28b627c44ca24"
format_stamp: "Formatted at 2026-09-28 14:53:16 on dist-test-slave-vk40"
server_key: "a267e2293565d32a8f12144b3882e04d"
server_key_iv: "2774a670f4e0ba88674bfb7d6ce810e6"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:16.384858 27890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 14:53:16.400369 27912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:16.401971 27890 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260928 14:53:16.402145 27890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "3bc4c56232214648a0b28b627c44ca24"
format_stamp: "Formatted at 2026-09-28 14:53:16 on dist-test-slave-vk40"
server_key: "a267e2293565d32a8f12144b3882e04d"
server_key_iv: "2774a670f4e0ba88674bfb7d6ce810e6"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:16.402377 27890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:16.426002 27890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:16.430194 27890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:16.430493 27890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:16.431495 27890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:16.433008 27890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:16.433141 27890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260928 14:53:16.433275 27890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:16.433328 27890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:16.462985 27890 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:43067
I20260928 14:53:16.463035 28024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:43067 every 8 connection(s)
I20260928 14:53:16.464298 27890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:53:16.467774 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 27890
I20260928 14:53:16.467964 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:53:16.468530 15303 external_mini_cluster.cc:1428] Setting key 884dc8031f4ff900a5383e6112a8ca67
Sep 28 14:53:16 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607196, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:16.485164 28027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:16.466880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:51109 (local address 127.14.241.254:46527)
0928 14:53:16.467349 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:16.467355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:16.468391 (+ 1036us) server_negotiation.cc:407] Connection header received
0928 14:53:16.470578 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:16.470584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:16.470666 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:16.470777 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:16.472186 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:16.473252 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:16.474881 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:16.475454 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:16.475571 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:16.479139 (+ 3568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:16.479144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:16.479148 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:16.479192 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:16.481773 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:16.482689 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:16.482695 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:16.482698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:16.482825 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:16.483298 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:16.483302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:16.483304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:16.483578 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:16.483687 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:16.484648 (+ 961us) server_negotiation.cc:299] Negotiation successful
0928 14:53:16.484876 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":210,"threads_started":1}
I20260928 14:53:16.486593 28028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:16.467402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46527 (local address 127.14.241.194:51109)
0928 14:53:16.468216 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:16.468256 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:16.470278 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:16.470972 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:16.470981 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:16.471440 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:16.472015 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:16.472032 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:16.473395 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:16.473398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:16.474731 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:16.474750 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:16.475670 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:16.475675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:16.475887 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:16.476597 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:16.476626 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:16.478878 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:16.481934 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:16.481944 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:16.481962 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:16.482569 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:16.482922 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:16.482927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:16.482930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:16.483194 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:16.483687 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:16.483695 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:16.484114 (+ 419us) client_negotiation.cc:769] Sending connection context
0928 14:53:16.484556 (+ 442us) client_negotiation.cc:240] Negotiation successful
0928 14:53:16.485083 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":324,"threads_started":1}
I20260928 14:53:16.488114 28025 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46527
I20260928 14:53:16.488427 28025 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:16.489102 28025 heartbeater.cc:511] Master 127.14.241.254:46527 requested a full tablet report, sending...
I20260928 14:53:16.490528 27669 ts_manager.cc:194] Registered new tserver with Master: 3bc4c56232214648a0b28b627c44ca24 (127.14.241.194:43067)
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
I20260928 14:53:16.491334 27669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:51109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:53:16.519644 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:47345/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46527
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:16.843763 28031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:16.844062 28031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:16.844200 28031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:16.851639 28031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:16.851753 28031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:16.851904 28031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:53:16.861791 28031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.241.212:47345/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:46527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:16.863010 28031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:16.864218 28031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:16.877137 28036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:16.877228 28037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:16.877100 28039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:16.877847 28031 server_base.cc:1034] running on GCE node
I20260928 14:53:16.878473 28031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:16.879446 28031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:16.880654 28031 hybrid_clock.cc:648] HybridClock initialized: now 1790607196880590 us; error 52 us; skew 500 ppm
I20260928 14:53:16.881006 28031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:16 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607196, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:16.884187 28031 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:53:16.885802 28031 webserver.cc:460] Webserver started at http://127.14.241.195:40021/ using document root <none> and password file <none>
I20260928 14:53:16.886479 28031 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:16.886586 28031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:16.886884 28031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:53:16 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607196, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:53:16.902808 28031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "81fe3652174c4514b9ccd40c89e7e31d"
format_stamp: "Formatted at 2026-09-28 14:53:16 on dist-test-slave-vk40"
server_key: "5669c47486910144a39534562035b708"
server_key_iv: "27c00aa5d57d0090f4ec1231844f9a6a"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:16.903911 28031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "81fe3652174c4514b9ccd40c89e7e31d"
format_stamp: "Formatted at 2026-09-28 14:53:16 on dist-test-slave-vk40"
server_key: "5669c47486910144a39534562035b708"
server_key_iv: "27c00aa5d57d0090f4ec1231844f9a6a"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:16.907718 27884 heartbeater.cc:503] Master 127.14.241.254:46527 was elected leader, sending a full tablet report...
I20260928 14:53:16.911404 28031 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260928 14:53:16.929350 28046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:16.931201 28031 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260928 14:53:16.931396 28031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "81fe3652174c4514b9ccd40c89e7e31d"
format_stamp: "Formatted at 2026-09-28 14:53:16 on dist-test-slave-vk40"
server_key: "5669c47486910144a39534562035b708"
server_key_iv: "27c00aa5d57d0090f4ec1231844f9a6a"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:16.931661 28031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:16.981366 28031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:16.985076 28031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:16.985334 28031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:16.986269 28031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:16.987707 28031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:16.987778 28031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:16.987900 28031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:16.987948 28031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:17.017999 28031 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:45655
I20260928 14:53:17.018024 28158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:45655 every 8 connection(s)
I20260928 14:53:17.019297 28031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:53:17.022053 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 28031
I20260928 14:53:17.022234 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:53:17.022691 15303 external_mini_cluster.cc:1428] Setting key 7c43ee5eacbb2b6e89bf1e7c0a1f9d22
Sep 28 14:53:17 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607196, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:17.039168 28161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:17.021948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:44185 (local address 127.14.241.254:46527)
0928 14:53:17.022455 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:17.022461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:17.023424 (+ 963us) server_negotiation.cc:407] Connection header received
0928 14:53:17.025640 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:17.025645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:17.025744 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:17.025836 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:17.027301 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:17.028022 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:17.029671 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:17.030096 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:17.030205 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:17.033506 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:17.033512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:17.033517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:17.033581 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:17.035971 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:17.036752 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:17.036757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:17.036761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:17.036916 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:17.037422 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:17.037428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:17.037431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:17.037826 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:17.037938 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:17.038744 (+ 806us) server_negotiation.cc:299] Negotiation successful
0928 14:53:17.038913 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":236,"threads_started":1}
I20260928 14:53:17.040122 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:17.022424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46527 (local address 127.14.241.195:44185)
0928 14:53:17.023250 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:17.023302 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:17.025383 (+ 2081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:17.026096 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:17.026104 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:17.026526 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:17.027133 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:17.027152 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:17.028165 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:17.028171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:17.029502 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:17.029543 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:17.030314 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:17.030321 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:17.030578 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:17.031168 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:17.031189 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:17.033288 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:17.036105 (+ 2817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:17.036115 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:17.036133 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:17.036623 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:17.037029 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:17.037034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:17.037037 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:17.037299 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:17.037947 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:17.037955 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:17.038402 (+ 447us) client_negotiation.cc:769] Sending connection context
0928 14:53:17.038633 (+ 231us) client_negotiation.cc:240] Negotiation successful
0928 14:53:17.039019 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":319,"threads_started":1}
I20260928 14:53:17.041442 28159 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46527
I20260928 14:53:17.041730 28159 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:17.042395 28159 heartbeater.cc:511] Master 127.14.241.254:46527 requested a full tablet report, sending...
I20260928 14:53:17.043713 27669 ts_manager.cc:194] Registered new tserver with Master: 81fe3652174c4514b9ccd40c89e7e31d (127.14.241.195:45655)
I20260928 14:53:17.044605 27669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:44185
I20260928 14:53:17.050355 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:53:17.051959 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 27749
Sep 28 14:53:17 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:53:17.068496 28164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:17.052802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59438 (local address 127.14.241.193:36201)
0928 14:53:17.053351 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:17.053358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:17.053383 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:53:17.053492 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:17.053499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:17.053693 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:17.053824 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:17.055237 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:17.056408 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:17.057904 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:17.058483 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:17.058581 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:17.062625 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:17.062639 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:17.062654 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:17.062698 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:17.065488 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:17.066241 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:17.066248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:17.066250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:17.066401 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:17.066818 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:17.066824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:17.066827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:17.067317 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:17.067627 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:17.067894 (+ 267us) server_negotiation.cc:299] Negotiation successful
0928 14:53:17.068117 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":276,"threads_started":1}
I20260928 14:53:17.069746 27879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:17.154719 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 27749
I20260928 14:53:17.167315 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 27890
Sep 28 14:53:17 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:53:17.183512 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:17.168240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47100 (local address 127.14.241.194:43067)
0928 14:53:17.168799 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:17.168806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:17.168830 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:53:17.168889 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:17.168895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:17.169108 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:17.169241 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:17.170465 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:17.171420 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:17.172817 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:17.173409 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:17.173517 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:17.177714 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:17.177728 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:17.177744 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:17.177786 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:17.180672 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:17.181393 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:17.181401 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:17.181404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:17.181536 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:17.181912 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:17.181916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:17.181918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:17.182284 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:17.182592 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:17.182898 (+ 306us) server_negotiation.cc:299] Negotiation successful
0928 14:53:17.183157 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":295,"threads_started":1}
I20260928 14:53:17.184801 28020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:17.266104 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 27890
I20260928 14:53:17.278199 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 28031
Sep 28 14:53:17 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:53:17.293524 28168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:17.279203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39452 (local address 127.14.241.195:45655)
0928 14:53:17.279809 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:17.279815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:17.279843 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:53:17.279910 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:17.279916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:17.280126 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:17.280248 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:17.281545 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:17.282545 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:17.283776 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:17.284370 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:17.284463 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:17.288444 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:17.288456 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:17.288472 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:17.288512 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:17.291186 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:17.291783 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:17.291790 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:17.291792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:17.291937 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:17.292213 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:17.292216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:17.292218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:17.292580 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:17.292814 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:17.293046 (+ 232us) server_negotiation.cc:299] Negotiation successful
0928 14:53:17.293238 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":323,"threads_started":1}
I20260928 14:53:17.294785 28154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:17.377488 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 28031
I20260928 14:53:17.389820 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 27637
I20260928 14:53:17.391099 27699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:17.496335 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 27637
I20260928 14:53:17.513625 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46527
--webserver_interface=127.14.241.254
--webserver_port=45559
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:46527
--ranger_config_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:47345/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.241.212:47345/kms with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:17.828397 28171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:17.828693 28171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:17.828788 28171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:17.836315 28171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:53:17.836421 28171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:17.836472 28171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:17.836526 28171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:53:17.836571 28171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:53:17.846731 28171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.241.212:47345/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:46527
--ranger_config_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=45559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:17.847975 28171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:17.849390 28171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:17.861160 28176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:17.861238 28177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:17.861136 28179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:17.861836 28171 server_base.cc:1034] running on GCE node
I20260928 14:53:17.862655 28171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:17.864205 28171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:17.865427 28171 hybrid_clock.cc:648] HybridClock initialized: now 1790607197865373 us; error 46 us; skew 500 ppm
I20260928 14:53:17.865767 28171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:17 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607197, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:17.868839 28171 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:53:17.870297 28171 webserver.cc:460] Webserver started at http://127.14.241.254:45559/ using document root <none> and password file <none>
I20260928 14:53:17.870944 28171 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:17.871061 28171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:53:17 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607197, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:53:17.945879 28171 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 14:53:17.948930 28187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:17.950335 28171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:53:17.950516 28171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4"
format_stamp: "Formatted at 2026-09-28 14:53:13 on dist-test-slave-vk40"
server_key: "c58668c674ab79ce58e3e2b4b60941be"
server_key_iv: "1c3a96e81b752c0c6467e63e2c1e7f13"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:17.951496 28171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:17.972867 28171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:17.976881 28171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:17.977152 28171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:17.994509 28171 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:46527
I20260928 14:53:17.994529 28238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:46527 every 8 connection(s)
I20260928 14:53:17.996003 28171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:53:18.002825 28239 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:53:18.006024 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 28171
I20260928 14:53:18.006515 28239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4: Bootstrap starting.
I20260928 14:53:18.007081 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:47345/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:36201
--local_ip_for_outbound_sockets=127.14.241.193
--tserver_master_addrs=127.14.241.254:46527
--webserver_port=39173
--webserver_interface=127.14.241.193
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.241.212:47345/kms with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:53:18.027266 28239 log.cc:824] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:18.036435 28239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:53:18.036927 28239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4: Bootstrap complete.
I20260928 14:53:18.045830 28239 raft_consensus.cc:348] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } }
I20260928 14:53:18.046655 28239 raft_consensus.cc:729] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec18ba7ae5b45eeb4040b746c3bd6e4, State: Initialized, Role: FOLLOWER
I20260928 14:53:18.047254 28239 consensus_queue.cc:260] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } }
I20260928 14:53:18.047463 28239 raft_consensus.cc:388] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:53:18.047574 28239 raft_consensus.cc:482] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:53:18.047731 28239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:53:18.051501 28239 raft_consensus.cc:504] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } }
I20260928 14:53:18.051991 28239 leader_election.cc:302] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cec18ba7ae5b45eeb4040b746c3bd6e4; no voters:
I20260928 14:53:18.052609 28239 leader_election.cc:288] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:53:18.052745 28243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:53:18.053491 28243 raft_consensus.cc:686] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [term 2 LEADER]: Becoming Leader. State: Replica: cec18ba7ae5b45eeb4040b746c3bd6e4, State: Running, Role: LEADER
I20260928 14:53:18.054055 28243 consensus_queue.cc:237] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } }
I20260928 14:53:18.054769 28239 sys_catalog.cc:564] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:53:18.057555 28245 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } } }
I20260928 14:53:18.057742 28244 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec18ba7ae5b45eeb4040b746c3bd6e4. Latest consensus state: current_term: 2 leader_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec18ba7ae5b45eeb4040b746c3bd6e4" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46527 } } }
I20260928 14:53:18.058279 28244 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:18.057991 28245 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:18.062845 28249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:53:18.068428 28249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:53:18.069906 28249 catalog_manager.cc:1259] Loaded cluster ID: 69a202bbcdab47e3b0ad08ef299c1f73
I20260928 14:53:18.069999 28249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:53:18.074325 28249 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:53:18.075871 28249 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4: Loaded TSK: 0
I20260928 14:53:18.077593 28249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:53:18.086130 28239 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260928 14:53:18.459287 28241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:18.459740 28241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:18.459903 28241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:18.471424 28241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:18.471585 28241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:18.471786 28241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:53:18.487301 28241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.241.212:47345/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:36201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=39173
--tserver_master_addrs=127.14.241.254:46527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:18.489051 28241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:18.490854 28241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:18.520584 28276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:18.520670 28277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:18.536422 28241 server_base.cc:1034] running on GCE node
W20260928 14:53:18.539538 28279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:18.541296 28241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:18.551754 28241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:18.553452 28241 hybrid_clock.cc:648] HybridClock initialized: now 1790607198553362 us; error 107 us; skew 500 ppm
I20260928 14:53:18.553923 28241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:18 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607198, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:18.559409 28241 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:53:18.592746 28241 webserver.cc:460] Webserver started at http://127.14.241.193:39173/ using document root <none> and password file <none>
I20260928 14:53:18.593920 28241 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:18.594162 28241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:53:18 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607198, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:53:18.621505 28241 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:53:18.627661 28288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:18.629446 28241 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260928 14:53:18.629642 28241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "782e83676db4470d9e453c99f846df55"
format_stamp: "Formatted at 2026-09-28 14:53:15 on dist-test-slave-vk40"
server_key: "bb3aed8fa43b3b30401cab47d106f37f"
server_key_iv: "10aab69eaf1890c994e72caab6cda018"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:18.630842 28241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:18.651695 28241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:18.655977 28241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:18.656258 28241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:18.657593 28241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:18.659040 28241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:18.659138 28241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:18.659252 28241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:18.659319 28241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:18.705856 28241 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:36201
I20260928 14:53:18.705906 28400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:36201 every 8 connection(s)
I20260928 14:53:18.707856 28241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:53:18.713336 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 28241
I20260928 14:53:18.714126 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:47345/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:43067
--local_ip_for_outbound_sockets=127.14.241.194
--tserver_master_addrs=127.14.241.254:46527
--webserver_port=42127
--webserver_interface=127.14.241.194
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.241.212:47345/kms with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:53:18 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607198, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:18.738930 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:18.711219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46527 (local address 127.14.241.193:58603)
0928 14:53:18.711993 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:18.712054 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:18.714970 (+ 2916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:18.717177 (+ 2207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:18.717189 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:18.717819 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:18.718626 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:18.718652 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:18.721154 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:18.721159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:18.722975 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:18.722993 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:18.724988 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:18.724996 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:18.725294 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:18.726157 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:18.726189 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:18.728902 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:18.733478 (+ 4576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:18.733488 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:18.733507 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:18.734181 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:18.734847 (+ 666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:18.734852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:18.734856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:18.735141 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:18.736207 (+ 1066us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:18.736215 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:18.736614 (+ 399us) client_negotiation.cc:769] Sending connection context
0928 14:53:18.736871 (+ 257us) client_negotiation.cc:240] Negotiation successful
0928 14:53:18.737501 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":992,"threads_started":1}
I20260928 14:53:18.738932 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:18.712422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:58603 (local address 127.14.241.254:46527)
0928 14:53:18.713582 (+ 1160us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:18.713601 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:18.713655 (+ 54us) server_negotiation.cc:407] Connection header received
0928 14:53:18.715758 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:18.715781 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:18.716376 (+ 595us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:18.716878 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:18.718877 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:18.720947 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:18.723206 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:18.724349 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:18.724555 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:18.729292 (+ 4737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:18.729303 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:18.729321 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:18.729379 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:18.733243 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:18.734380 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:18.734390 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:18.734400 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:18.734726 (+ 326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:18.735385 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:18.735389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:18.735391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:18.736047 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:18.736238 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:18.737126 (+ 888us) server_negotiation.cc:299] Negotiation successful
0928 14:53:18.737517 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":892,"thread_start_us":1140,"threads_started":1}
I20260928 14:53:18.742579 28401 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46527
I20260928 14:53:18.742913 28401 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:18.743650 28401 heartbeater.cc:511] Master 127.14.241.254:46527 requested a full tablet report, sending...
W20260928 14:53:18.746465 28401 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:46527 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 14:53:18.746690 28401 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:18.746955 28401 heartbeater.cc:511] Master 127.14.241.254:46527 requested a full tablet report, sending...
I20260928 14:53:18.748171 28401 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:18.748492 28401 heartbeater.cc:511] Master 127.14.241.254:46527 requested a full tablet report, sending...
W20260928 14:53:18.749507 28401 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:53:19.141023 28405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:19.141321 28405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:19.141463 28405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:19.148973 28405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:19.149103 28405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:19.149258 28405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:53:19.160460 28405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.241.212:47345/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:43067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=42127
--tserver_master_addrs=127.14.241.254:46527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:19.161813 28405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:19.163129 28405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:19.178272 28419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:19.178303 28422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:19.178337 28420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:19.179557 28405 server_base.cc:1034] running on GCE node
I20260928 14:53:19.180222 28405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:19.181303 28405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:19.182507 28405 hybrid_clock.cc:648] HybridClock initialized: now 1790607199182431 us; error 68 us; skew 500 ppm
I20260928 14:53:19.182906 28405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:19 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607199, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:19.187288 28405 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:53:19.197050 28405 webserver.cc:460] Webserver started at http://127.14.241.194:42127/ using document root <none> and password file <none>
I20260928 14:53:19.198135 28405 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:19.198314 28405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:53:19 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607199, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:53:19.241984 28405 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.005s sys 0.000s
I20260928 14:53:19.245886 28432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:19.247768 28405 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260928 14:53:19.247965 28405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "3bc4c56232214648a0b28b627c44ca24"
format_stamp: "Formatted at 2026-09-28 14:53:16 on dist-test-slave-vk40"
server_key: "a267e2293565d32a8f12144b3882e04d"
server_key_iv: "2774a670f4e0ba88674bfb7d6ce810e6"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:19.249550 28405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:19.276561 28405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:19.282238 28405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:19.282577 28405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:19.283838 28405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:19.285619 28405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:19.285729 28405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:19.285835 28405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:19.285907 28405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:53:19 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790607199, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:19.331015 28405 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:43067
I20260928 14:53:19.331050 28544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:43067 every 8 connection(s)
I20260928 14:53:19.332690 28405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:53:19.342267 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 28405
I20260928 14:53:19.343055 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.241.212:47345/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:45655
--local_ip_for_outbound_sockets=127.14.241.195
--tserver_master_addrs=127.14.241.254:46527
--webserver_port=40021
--webserver_interface=127.14.241.195
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.241.212:47345/kms with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:53:19 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607199, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:19.370007 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:19.348510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:43271 (local address 127.14.241.254:46527)
0928 14:53:19.350060 (+ 1550us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:19.350066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:19.350085 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:53:19.350147 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:19.350152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:19.350241 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:19.350345 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:19.354312 (+ 3967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:19.355614 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:19.358327 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:19.359045 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:19.359153 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:19.363642 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:19.363648 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:19.363652 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:19.363699 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:19.366647 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:19.367515 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:19.367522 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:19.367525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:19.367695 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:19.368149 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:19.368153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:19.368155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:19.368496 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:19.368634 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:19.369405 (+ 771us) server_negotiation.cc:299] Negotiation successful
0928 14:53:19.369640 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1430,"thread_start_us":1315,"threads_started":1}
I20260928 14:53:19.371300 28547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:19.342400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46527 (local address 127.14.241.194:43271)
0928 14:53:19.346317 (+ 3917us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:19.346381 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:19.348710 (+ 2329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:19.352471 (+ 3761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:19.352485 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:19.353069 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:19.354041 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:19.354068 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:19.355878 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:19.355884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:19.358117 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:19.358135 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:19.359553 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:19.359560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:19.359889 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:19.360706 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:19.360761 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:19.363363 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:19.366850 (+ 3487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:19.366861 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:19.366878 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:19.367364 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:19.367812 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:19.367818 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:19.367822 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:19.368024 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:19.368622 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:19.368631 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:19.369022 (+ 391us) client_negotiation.cc:769] Sending connection context
0928 14:53:19.369274 (+ 252us) client_negotiation.cc:240] Negotiation successful
0928 14:53:19.369864 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3575,"thread_start_us":3348,"threads_started":1}
I20260928 14:53:19.372915 28545 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46527
I20260928 14:53:19.373214 28545 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:19.373936 28545 heartbeater.cc:511] Master 127.14.241.254:46527 requested a full tablet report, sending...
I20260928 14:53:19.375644 28204 ts_manager.cc:194] Registered new tserver with Master: 3bc4c56232214648a0b28b627c44ca24 (127.14.241.194:43067)
I20260928 14:53:19.377599 28204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:43271
I20260928 14:53:19.749668 28401 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:19.750133 28401 heartbeater.cc:511] Master 127.14.241.254:46527 requested a full tablet report, sending...
I20260928 14:53:19.751395 28204 ts_manager.cc:194] Registered new tserver with Master: 782e83676db4470d9e453c99f846df55 (127.14.241.193:36201)
I20260928 14:53:19.752594 28204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:58603
Sep 28 14:53:19 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607199, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
W20260928 14:53:19.839986 28549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:19.840456 28549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:19.840626 28549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:19.851501 28549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:19.851662 28549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:19.851856 28549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:53:19.867096 28549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.241.212:47345/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:45655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=40021
--tserver_master_addrs=127.14.241.254:46527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:19.868837 28549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:19.870561 28549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:19.936787 28555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:19.936995 28556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:19.956053 28549 server_base.cc:1034] running on GCE node
W20260928 14:53:19.957310 28558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:19.958232 28549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:19.965487 28549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:19.966758 28549 hybrid_clock.cc:648] HybridClock initialized: now 1790607199966689 us; error 62 us; skew 500 ppm
I20260928 14:53:19.967157 28549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:19 dist-test-slave-vk40 krb5kdc[27193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607199, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:19.971345 28549 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:53:19.998667 28549 webserver.cc:460] Webserver started at http://127.14.241.195:40021/ using document root <none> and password file <none>
I20260928 14:53:19.999711 28549 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:19.999879 28549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:53:20 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607199, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:53:20.074455 28549 fs_manager.cc:709] Time spent opening directory manager: real 0.027s user 0.003s sys 0.003s
I20260928 14:53:20.088999 28566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:20.100654 28549 fs_manager.cc:725] Time spent opening block manager: real 0.024s user 0.005s sys 0.001s
I20260928 14:53:20.100872 28549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "81fe3652174c4514b9ccd40c89e7e31d"
format_stamp: "Formatted at 2026-09-28 14:53:16 on dist-test-slave-vk40"
server_key: "5669c47486910144a39534562035b708"
server_key_iv: "27c00aa5d57d0090f4ec1231844f9a6a"
server_key_version: "kuduclusterkey@0"
I20260928 14:53:20.102339 28549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:20.156522 28549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:20.161710 28549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:20.162056 28549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:20.164732 28549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:20.173154 28549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:20.173343 28549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:20.173509 28549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:20.173615 28549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:53:20 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607199, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for HTTP/127.14.241.212@KRBTEST.COM
I20260928 14:53:20.381560 28545 heartbeater.cc:503] Master 127.14.241.254:46527 was elected leader, sending a full tablet report...
I20260928 14:53:20.448665 28549 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:45655
I20260928 14:53:20.448726 28679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:45655 every 8 connection(s)
I20260928 14:53:20.450719 28549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:53:20.452823 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 28549
Sep 28 14:53:20 dist-test-slave-vk40 krb5kdc[27193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607199, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:20.492852 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.454547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51776 (local address 127.14.241.254:46527)
0928 14:53:20.474739 (+ 20192us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:20.474746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:20.475502 (+ 756us) server_negotiation.cc:407] Connection header received
0928 14:53:20.475580 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:20.475586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:20.475690 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:20.475800 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:20.477940 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.478961 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.480892 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.481524 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.481634 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.485535 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:20.485542 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:20.485547 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:20.485593 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:20.488942 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:20.489807 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:20.489813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:20.489816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:20.489959 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:20.490391 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:20.490396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:20.490399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:20.490753 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:20.491016 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:20.492189 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0928 14:53:20.492444 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20057,"thread_start_us":19958,"threads_started":1}
I20260928 14:53:20.501368 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.465644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:50621 (local address 127.14.241.254:46527)
0928 14:53:20.476809 (+ 11165us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:20.476815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:20.476835 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:53:20.476895 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:20.476900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:20.476979 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:20.477095 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:20.479393 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.480854 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.483322 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.484021 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.484119 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.493807 (+ 9688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:20.493815 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:20.493820 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:20.493857 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:20.498231 (+ 4374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:20.499021 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:20.499027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:20.499031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:20.499148 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:20.499586 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:20.499592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:20.499595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:20.499866 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:20.499988 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:20.500763 (+ 775us) server_negotiation.cc:299] Negotiation successful
0928 14:53:20.500988 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11010,"thread_start_us":12010,"threads_started":1}
I20260928 14:53:20.502436 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.466377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46527 (local address 127.14.241.195:50621)
0928 14:53:20.473895 (+ 7518us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:20.473948 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:20.476322 (+ 2374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:20.477601 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:20.477615 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:20.478255 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:20.479135 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.479171 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.481040 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.481046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.483140 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.483158 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.484536 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.484544 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.484861 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.490465 (+ 5604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:20.490499 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:20.493504 (+ 3005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:20.498372 (+ 4868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:20.498380 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:20.498395 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:20.498899 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:20.499263 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:20.499266 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:20.499269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:20.499470 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:20.500001 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:20.500009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:20.500409 (+ 400us) client_negotiation.cc:769] Sending connection context
0928 14:53:20.500666 (+ 257us) client_negotiation.cc:240] Negotiation successful
0928 14:53:20.501250 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7202,"thread_start_us":6965,"threads_started":1}
I20260928 14:53:20.503796 28680 heartbeater.cc:347] Connected to a master server at 127.14.241.254:46527
I20260928 14:53:20.504081 28680 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:20.504901 28680 heartbeater.cc:511] Master 127.14.241.254:46527 requested a full tablet report, sending...
I20260928 14:53:20.506955 28203 ts_manager.cc:194] Registered new tserver with Master: 81fe3652174c4514b9ccd40c89e7e31d (127.14.241.195:45655)
I20260928 14:53:20.508204 28203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:50621
I20260928 14:53:20.511835 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:53:20.528507 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.515454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51786 (local address 127.14.241.254:46527)
0928 14:53:20.515617 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:20.515623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:20.516065 (+ 442us) server_negotiation.cc:407] Connection header received
0928 14:53:20.516247 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:20.516253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:20.516375 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:20.516459 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:20.517959 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.518691 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.520098 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.520627 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.520742 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.523317 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:20.523322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:20.523325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:20.523360 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:20.525882 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:20.526538 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:20.526543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:20.526545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:20.526687 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:20.527076 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:20.527080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:20.527082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:20.527363 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:20.527582 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:20.527919 (+ 337us) server_negotiation.cc:299] Negotiation successful
0928 14:53:20.528152 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 14:53:20.533921 28203 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:53:20.541546 28203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:53:20.586724 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.556198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:36201 (local address 127.0.0.1:59448)
0928 14:53:20.577394 (+ 21196us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:20.577455 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:20.577696 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:20.578658 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:20.578666 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:20.578697 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:20.579148 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.579165 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.581252 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.581259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.583582 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.583602 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.585739 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.585746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.585936 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.585990 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 14:53:20.586111 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 14:53:20.586304 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20920,"spinlock_wait_cycles":8576,"thread_start_us":20119,"threads_started":1}
I20260928 14:53:20.588073 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.556687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59448 (local address 127.14.241.193:36201)
0928 14:53:20.576814 (+ 20127us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:20.576824 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:20.577956 (+ 1132us) server_negotiation.cc:407] Connection header received
0928 14:53:20.578070 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:20.578078 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:20.578409 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:20.578626 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:20.579399 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.581061 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.583787 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.585176 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.585370 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.585573 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:20.587586 (+ 2013us) server_negotiation.cc:299] Negotiation successful
0928 14:53:20.587759 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19921,"thread_start_us":19751,"threads_started":1}
I20260928 14:53:20.588773 28706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.556073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:45655 (local address 127.0.0.1:39458)
0928 14:53:20.577972 (+ 21899us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:20.577988 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:20.578093 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:20.580966 (+ 2873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:20.580972 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:20.581000 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:20.581357 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.581370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.583260 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.583265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.585362 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.585375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.587337 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.587343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.587526 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.587563 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 14:53:20.587628 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:53:20.588361 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21005,"mutex_wait_us":575,"thread_start_us":20949,"threads_started":1}
I20260928 14:53:20.591811 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.556737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:43067 (local address 127.0.0.1:47112)
0928 14:53:20.584007 (+ 27270us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:20.584027 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:20.584152 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:20.585092 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:20.585097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:20.585121 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:20.585522 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.585533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.587344 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.587348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.589320 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.589337 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.591065 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.591071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.591241 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.591272 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 14:53:20.591355 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:53:20.591479 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27135,"thread_start_us":34434,"threads_started":1}
I20260928 14:53:20.592510 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.557207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47112 (local address 127.14.241.194:43067)
0928 14:53:20.583388 (+ 26181us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:20.583397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:20.584307 (+ 910us) server_negotiation.cc:407] Connection header received
0928 14:53:20.584420 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:20.584428 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:20.584676 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:20.584853 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:20.585717 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.587197 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.589503 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.590843 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.590995 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.591143 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:20.592005 (+ 862us) server_negotiation.cc:299] Negotiation successful
0928 14:53:20.592160 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25998,"thread_start_us":26200,"threads_started":1}
I20260928 14:53:20.592866 28703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.556647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.14.241.195:45655)
0928 14:53:20.579848 (+ 23201us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:20.579858 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:20.579893 (+ 35us) server_negotiation.cc:407] Connection header received
0928 14:53:20.579988 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:20.579995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:20.580246 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:20.580662 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:20.581572 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.583102 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.585599 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.587090 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.587886 (+ 796us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.588068 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:20.588138 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 14:53:20.589215 (+ 1077us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22989,"thread_start_us":23627,"threads_started":1}
I20260928 14:53:20.592577 28335 tablet_service.cc:1467] Processing CreateTablet for tablet 477e389053094f60bbf90b4d8bb129e8 (DEFAULT_TABLE table=test-table [id=310b175562d64f9a948052990873a44f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:20.594205 28335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477e389053094f60bbf90b4d8bb129e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:20.596704 28480 tablet_service.cc:1467] Processing CreateTablet for tablet 477e389053094f60bbf90b4d8bb129e8 (DEFAULT_TABLE table=test-table [id=310b175562d64f9a948052990873a44f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:20.598261 28480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477e389053094f60bbf90b4d8bb129e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:20.598927 28615 tablet_service.cc:1467] Processing CreateTablet for tablet 477e389053094f60bbf90b4d8bb129e8 (DEFAULT_TABLE table=test-table [id=310b175562d64f9a948052990873a44f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:20.600383 28615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477e389053094f60bbf90b4d8bb129e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:20.615953 28708 tablet_bootstrap.cc:492] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55: Bootstrap starting.
I20260928 14:53:20.619436 28709 tablet_bootstrap.cc:492] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24: Bootstrap starting.
I20260928 14:53:20.621629 28708 tablet_bootstrap.cc:654] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:20.623121 28708 log.cc:824] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:20.623950 28709 tablet_bootstrap.cc:654] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:20.625187 28709 log.cc:824] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:20.628532 28710 tablet_bootstrap.cc:492] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: Bootstrap starting.
I20260928 14:53:20.635843 28710 tablet_bootstrap.cc:654] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:20.636989 28708 tablet_bootstrap.cc:492] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55: No bootstrap required, opened a new log
I20260928 14:53:20.637203 28710 log.cc:824] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:20.637336 28708 ts_tablet_manager.cc:1397] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.004s
I20260928 14:53:20.647061 28709 tablet_bootstrap.cc:492] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24: No bootstrap required, opened a new log
I20260928 14:53:20.647454 28709 ts_tablet_manager.cc:1397] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24: Time spent bootstrapping tablet: real 0.028s user 0.013s sys 0.013s
I20260928 14:53:20.647759 28710 tablet_bootstrap.cc:492] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: No bootstrap required, opened a new log
I20260928 14:53:20.648078 28710 ts_tablet_manager.cc:1397] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.009s
I20260928 14:53:20.648393 28708 raft_consensus.cc:348] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782e83676db4470d9e453c99f846df55" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 36201 } } peers { permanent_uuid: "81fe3652174c4514b9ccd40c89e7e31d" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 45655 } } peers { permanent_uuid: "3bc4c56232214648a0b28b627c44ca24" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43067 } }
I20260928 14:53:20.648980 28708 raft_consensus.cc:374] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:20.649127 28708 raft_consensus.cc:729] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 782e83676db4470d9e453c99f846df55, State: Initialized, Role: FOLLOWER
I20260928 14:53:20.649916 28708 consensus_queue.cc:260] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782e83676db4470d9e453c99f846df55" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 36201 } } peers { permanent_uuid: "81fe3652174c4514b9ccd40c89e7e31d" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 45655 } } peers { permanent_uuid: "3bc4c56232214648a0b28b627c44ca24" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43067 } }
I20260928 14:53:20.658205 28708 ts_tablet_manager.cc:1428] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55: Time spent starting tablet: real 0.021s user 0.015s sys 0.004s
I20260928 14:53:20.658123 28709 raft_consensus.cc:348] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782e83676db4470d9e453c99f846df55" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 36201 } } peers { permanent_uuid: "81fe3652174c4514b9ccd40c89e7e31d" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 45655 } } peers { permanent_uuid: "3bc4c56232214648a0b28b627c44ca24" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43067 } }
I20260928 14:53:20.658643 28401 heartbeater.cc:503] Master 127.14.241.254:46527 was elected leader, sending a full tablet report...
I20260928 14:53:20.658658 28709 raft_consensus.cc:374] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:20.658746 28709 raft_consensus.cc:729] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc4c56232214648a0b28b627c44ca24, State: Initialized, Role: FOLLOWER
I20260928 14:53:20.659441 28709 consensus_queue.cc:260] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782e83676db4470d9e453c99f846df55" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 36201 } } peers { permanent_uuid: "81fe3652174c4514b9ccd40c89e7e31d" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 45655 } } peers { permanent_uuid: "3bc4c56232214648a0b28b627c44ca24" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43067 } }
I20260928 14:53:20.660162 28710 raft_consensus.cc:348] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782e83676db4470d9e453c99f846df55" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 36201 } } peers { permanent_uuid: "81fe3652174c4514b9ccd40c89e7e31d" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 45655 } } peers { permanent_uuid: "3bc4c56232214648a0b28b627c44ca24" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43067 } }
I20260928 14:53:20.660604 28710 raft_consensus.cc:374] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:20.660692 28710 raft_consensus.cc:729] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81fe3652174c4514b9ccd40c89e7e31d, State: Initialized, Role: FOLLOWER
I20260928 14:53:20.661374 28710 consensus_queue.cc:260] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782e83676db4470d9e453c99f846df55" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 36201 } } peers { permanent_uuid: "81fe3652174c4514b9ccd40c89e7e31d" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 45655 } } peers { permanent_uuid: "3bc4c56232214648a0b28b627c44ca24" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43067 } }
I20260928 14:53:20.667732 28709 ts_tablet_manager.cc:1428] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24: Time spent starting tablet: real 0.020s user 0.011s sys 0.007s
I20260928 14:53:20.668910 28680 heartbeater.cc:503] Master 127.14.241.254:46527 was elected leader, sending a full tablet report...
I20260928 14:53:20.669569 28710 ts_tablet_manager.cc:1428] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: Time spent starting tablet: real 0.021s user 0.014s sys 0.004s
I20260928 14:53:20.682075 28714 raft_consensus.cc:482] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:53:20.682402 28714 raft_consensus.cc:504] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782e83676db4470d9e453c99f846df55" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 36201 } } peers { permanent_uuid: "81fe3652174c4514b9ccd40c89e7e31d" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 45655 } } peers { permanent_uuid: "3bc4c56232214648a0b28b627c44ca24" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43067 } }
I20260928 14:53:20.684158 28714 leader_election.cc:288] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81fe3652174c4514b9ccd40c89e7e31d (127.14.241.195:45655), 3bc4c56232214648a0b28b627c44ca24 (127.14.241.194:43067)
I20260928 14:53:20.692019 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.684744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:34635 (local address 127.14.241.194:43067)
0928 14:53:20.684931 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:20.684937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:20.685855 (+ 918us) server_negotiation.cc:407] Connection header received
0928 14:53:20.685991 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:20.685996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:20.686068 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:20.686151 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:20.686732 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.687506 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.690130 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.690932 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.691076 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.691150 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:20.691632 (+ 482us) server_negotiation.cc:299] Negotiation successful
0928 14:53:20.691768 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 14:53:20.692166 28703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.684769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:49065 (local address 127.14.241.195:45655)
0928 14:53:20.684953 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:20.684958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:20.685532 (+ 574us) server_negotiation.cc:407] Connection header received
0928 14:53:20.685816 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:20.685822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:20.685900 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:20.685980 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:20.686651 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.687687 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.690112 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.691298 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.691453 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.691525 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:20.691804 (+ 279us) server_negotiation.cc:299] Negotiation successful
0928 14:53:20.691954 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 14:53:20.692471 28718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.685076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:43067 (local address 127.14.241.193:34635)
0928 14:53:20.685746 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:20.685762 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:20.685880 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:20.686170 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:20.686174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:20.686227 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:20.686568 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.686578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.687635 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.687640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.689934 (+ 2294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.689952 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.691295 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.691302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.691470 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.691506 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 14:53:20.691594 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 14:53:20.691728 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":73,"spinlock_wait_cycles":27520,"thread_start_us":467,"threads_started":1}
I20260928 14:53:20.692907 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.684682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:45655 (local address 127.14.241.193:49065)
0928 14:53:20.685419 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:20.685437 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:20.685653 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:20.686013 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:20.686017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:20.686040 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:20.686481 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.686497 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.687816 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.687820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.689938 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:20.689951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.691498 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.691503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:20.691647 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.691675 (+ 28us) client_negotiation.cc:769] Sending connection context
0928 14:53:20.692414 (+ 739us) client_negotiation.cc:240] Negotiation successful
0928 14:53:20.692551 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":19,"spinlock_wait_cycles":33152,"thread_start_us":573,"threads_started":1}
I20260928 14:53:20.693501 28500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477e389053094f60bbf90b4d8bb129e8" candidate_uuid: "782e83676db4470d9e453c99f846df55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc4c56232214648a0b28b627c44ca24" is_pre_election: true
I20260928 14:53:20.693816 28635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477e389053094f60bbf90b4d8bb129e8" candidate_uuid: "782e83676db4470d9e453c99f846df55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81fe3652174c4514b9ccd40c89e7e31d" is_pre_election: true
I20260928 14:53:20.694131 28500 raft_consensus.cc:2454] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 782e83676db4470d9e453c99f846df55 in term 0.
I20260928 14:53:20.694314 28635 raft_consensus.cc:2454] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 782e83676db4470d9e453c99f846df55 in term 0.
I20260928 14:53:20.694851 28292 leader_election.cc:302] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3bc4c56232214648a0b28b627c44ca24, 782e83676db4470d9e453c99f846df55, 81fe3652174c4514b9ccd40c89e7e31d; no voters:
I20260928 14:53:20.695251 28714 raft_consensus.cc:2781] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:53:20.695399 28714 raft_consensus.cc:482] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:53:20.695492 28714 raft_consensus.cc:3037] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:20.697751 28714 raft_consensus.cc:504] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782e83676db4470d9e453c99f846df55" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 36201 } } peers { permanent_uuid: "81fe3652174c4514b9ccd40c89e7e31d" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 45655 } } peers { permanent_uuid: "3bc4c56232214648a0b28b627c44ca24" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43067 } }
I20260928 14:53:20.698352 28714 leader_election.cc:288] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [CANDIDATE]: Term 1 election: Requested vote from peers 81fe3652174c4514b9ccd40c89e7e31d (127.14.241.195:45655), 3bc4c56232214648a0b28b627c44ca24 (127.14.241.194:43067)
I20260928 14:53:20.698879 28500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477e389053094f60bbf90b4d8bb129e8" candidate_uuid: "782e83676db4470d9e453c99f846df55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc4c56232214648a0b28b627c44ca24"
I20260928 14:53:20.698884 28635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477e389053094f60bbf90b4d8bb129e8" candidate_uuid: "782e83676db4470d9e453c99f846df55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81fe3652174c4514b9ccd40c89e7e31d"
I20260928 14:53:20.699075 28635 raft_consensus.cc:3037] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:20.699070 28500 raft_consensus.cc:3037] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:20.701733 28500 raft_consensus.cc:2454] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 782e83676db4470d9e453c99f846df55 in term 1.
I20260928 14:53:20.702343 28292 leader_election.cc:302] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc4c56232214648a0b28b627c44ca24, 782e83676db4470d9e453c99f846df55; no voters:
I20260928 14:53:20.702639 28714 raft_consensus.cc:2781] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:53:20.703123 28714 raft_consensus.cc:686] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [term 1 LEADER]: Becoming Leader. State: Replica: 782e83676db4470d9e453c99f846df55, State: Running, Role: LEADER
I20260928 14:53:20.703464 28635 raft_consensus.cc:2454] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 782e83676db4470d9e453c99f846df55 in term 1.
I20260928 14:53:20.703677 28714 consensus_queue.cc:237] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782e83676db4470d9e453c99f846df55" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 36201 } } peers { permanent_uuid: "81fe3652174c4514b9ccd40c89e7e31d" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 45655 } } peers { permanent_uuid: "3bc4c56232214648a0b28b627c44ca24" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43067 } }
I20260928 14:53:20.709005 28204 catalog_manager.cc:5495] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 782e83676db4470d9e453c99f846df55 (127.14.241.193). New cstate: current_term: 1 leader_uuid: "782e83676db4470d9e453c99f846df55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782e83676db4470d9e453c99f846df55" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 36201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81fe3652174c4514b9ccd40c89e7e31d" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 45655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bc4c56232214648a0b28b627c44ca24" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43067 } health_report { overall_health: UNKNOWN } } }
W20260928 14:53:20.712172 28402 tablet.cc:2369] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:53:20.714182 28681 tablet.cc:2369] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:53:20.739326 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.733018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.14.241.193:36201)
0928 14:53:20.733365 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:20.733371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:20.733409 (+ 38us) server_negotiation.cc:407] Connection header received
0928 14:53:20.733511 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:20.733516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:20.733620 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:20.733783 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:53:20.734264 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.735380 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.736946 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.737543 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.737660 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.738139 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:53:20.738797 (+ 658us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:53:20.738929 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:20.738972 (+ 43us) server_negotiation.cc:299] Negotiation successful
0928 14:53:20.739051 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
I20260928 14:53:20.748550 28500 raft_consensus.cc:1261] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24 [term 1 FOLLOWER]: Refusing update from remote peer 782e83676db4470d9e453c99f846df55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:53:20.748624 28635 raft_consensus.cc:1261] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d [term 1 FOLLOWER]: Refusing update from remote peer 782e83676db4470d9e453c99f846df55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:53:20.749497 28714 consensus_queue.cc:1035] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81fe3652174c4514b9ccd40c89e7e31d" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 45655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:53:20.749877 28719 consensus_queue.cc:1035] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc4c56232214648a0b28b627c44ca24" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:53:20.787658 28204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:53:20.787985 28204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:53:20.791641 28204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cec18ba7ae5b45eeb4040b746c3bd6e4: Sending DeleteTablet for 3 replicas of tablet 477e389053094f60bbf90b4d8bb129e8
I20260928 14:53:20.792960 28615 tablet_service.cc:1514] Processing DeleteTablet for tablet 477e389053094f60bbf90b4d8bb129e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:20 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:39458
I20260928 14:53:20.793687 28480 tablet_service.cc:1514] Processing DeleteTablet for tablet 477e389053094f60bbf90b4d8bb129e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:20 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:47112
I20260928 14:53:20.792966 28335 tablet_service.cc:1514] Processing DeleteTablet for tablet 477e389053094f60bbf90b4d8bb129e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:20 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59448
I20260928 14:53:20.798890 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 28241
I20260928 14:53:20.826839 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.802445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59466 (local address 127.14.241.193:36201)
0928 14:53:20.802641 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:20.802647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:20.802668 (+ 21us) server_negotiation.cc:407] Connection header received
0928 14:53:20.802735 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:20.802740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:20.802853 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:20.802975 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:20.805008 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.806296 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.808106 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.808874 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.809002 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.811921 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:20.811935 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:20.811955 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:20.812002 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:20.817457 (+ 5455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:20.821875 (+ 4418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:20.821885 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:20.821894 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:20.822106 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:20.823084 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:20.823090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:20.823093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:20.824044 (+ 951us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:20.824369 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:20.826077 (+ 1708us) server_negotiation.cc:299] Negotiation successful
0928 14:53:20.826356 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 14:53:20.827525 28396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:20.836090 28733 tablet_replica.cc:330] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24: stopping tablet replica
I20260928 14:53:20.836647 28733 raft_consensus.cc:2229] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:53:20.837230 28733 raft_consensus.cc:2258] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:53:20.838186 28732 tablet_replica.cc:330] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: stopping tablet replica
I20260928 14:53:20.838542 28732 raft_consensus.cc:2229] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:53:20.839793 28732 raft_consensus.cc:2258] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:53:20.840175 28733 ts_tablet_manager.cc:1905] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:53:20.842068 28732 ts_tablet_manager.cc:1905] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:53:20.848403 28733 ts_tablet_manager.cc:1918] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:53:20.848588 28733 log.cc:1196] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/477e389053094f60bbf90b4d8bb129e8
I20260928 14:53:20.848901 28732 ts_tablet_manager.cc:1918] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:53:20.849057 28732 log.cc:1196] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/477e389053094f60bbf90b4d8bb129e8
I20260928 14:53:20.849084 28733 ts_tablet_manager.cc:1939] T 477e389053094f60bbf90b4d8bb129e8 P 3bc4c56232214648a0b28b627c44ca24: Deleting consensus metadata
I20260928 14:53:20.849540 28732 ts_tablet_manager.cc:1939] T 477e389053094f60bbf90b4d8bb129e8 P 81fe3652174c4514b9ccd40c89e7e31d: Deleting consensus metadata
I20260928 14:53:20.850986 28191 catalog_manager.cc:4841] TS 3bc4c56232214648a0b28b627c44ca24 (127.14.241.194:43067): tablet 477e389053094f60bbf90b4d8bb129e8 (table test-table [id=310b175562d64f9a948052990873a44f]) successfully deleted
I20260928 14:53:20.851217 28189 catalog_manager.cc:4841] TS 81fe3652174c4514b9ccd40c89e7e31d (127.14.241.195:45655): tablet 477e389053094f60bbf90b4d8bb129e8 (table test-table [id=310b175562d64f9a948052990873a44f]) successfully deleted
I20260928 14:53:20.938031 28734 tablet_replica.cc:330] T 477e389053094f60bbf90b4d8bb129e8 P 782e83676db4470d9e453c99f846df55: stopping tablet replica
I20260928 14:53:20.938278 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 28241
W20260928 14:53:20.952451 28190 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.241.193:36201 (error 108)
I20260928 14:53:20.953208 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 28405
I20260928 14:53:20.954214 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.953750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:36201 (local address 127.0.0.1:59468)
0928 14:53:20.953961 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:20.954080 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:36201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260928 14:53:20.954424 28190 catalog_manager.cc:4584] TS 782e83676db4470d9e453c99f846df55 (127.14.241.193:36201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 477e389053094f60bbf90b4d8bb129e8: Network error: Client connection negotiation failed: client connection to 127.14.241.193:36201: connect: Connection refused (error 111)
I20260928 14:53:20.966801 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:20.953905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.14.241.194:43067)
0928 14:53:20.954071 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:20.954077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:20.954214 (+ 137us) server_negotiation.cc:407] Connection header received
0928 14:53:20.954433 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:20.954437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:20.954504 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:20.954626 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:20.956025 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.957042 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.958235 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:20.958619 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:20.958706 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:20.961075 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:20.961095 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:20.961112 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:20.961147 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:20.963902 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:20.964675 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:20.964683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:20.964685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:20.964828 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:20.965220 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:20.965224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:20.965226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:20.965603 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:20.965911 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:20.966205 (+ 294us) server_negotiation.cc:299] Negotiation successful
0928 14:53:20.966454 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:53:20.967458 28540 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:21.008245 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:21.007644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:36201 (local address 127.0.0.1:59476)
0928 14:53:21.007902 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:21.008053 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:36201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260928 14:53:21.009409 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:21.009035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:36201 (local address 127.0.0.1:59480)
0928 14:53:21.009197 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:21.009276 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:36201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260928 14:53:21.055891 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 28405
I20260928 14:53:21.069499 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 28549
I20260928 14:53:21.069938 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:21.069490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:36201 (local address 127.0.0.1:59486)
0928 14:53:21.069699 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:21.069776 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:36201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260928 14:53:21.071221 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:21.070877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:36201 (local address 127.0.0.1:59500)
0928 14:53:21.071038 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:21.071107 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:36201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 14:53:21.082291 28703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:21.070236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39468 (local address 127.14.241.195:45655)
0928 14:53:21.070379 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:21.070383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:21.070401 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:53:21.070582 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:21.070585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:21.070651 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:21.070809 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:21.072137 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:21.072921 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:21.074178 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:21.074612 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:21.074712 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:21.077140 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:21.077155 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:21.077172 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:21.077211 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:21.079755 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:21.080484 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:21.080491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:21.080494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:21.080625 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:21.080999 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:21.081002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:21.081004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:21.081372 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:21.081602 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:21.081830 (+ 228us) server_negotiation.cc:299] Negotiation successful
0928 14:53:21.082019 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 14:53:21.082934 28675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:21.151822 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:21.151214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:36201 (local address 127.0.0.1:59502)
0928 14:53:21.151485 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:21.151608 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:36201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260928 14:53:21.153192 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:21.152772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:36201 (local address 127.0.0.1:59518)
0928 14:53:21.152970 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:21.153054 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:36201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260928 14:53:21.176620 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 28549
I20260928 14:53:21.189787 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 28171
I20260928 14:53:21.190913 28234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:21.303632 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 28171
I20260928 14:53:21.321723 15303 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 14:53:21.656900 15303 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 14:53:21.657101 15303 mini_ranger.cc:66] Stopping Ranger...
I20260928 14:53:22.003396 15303 mini_ranger.cc:68] Stopped Ranger
2026-09-28 14:53:22.003 UTC [27223] LOG: received smart shutdown request
2026-09-28 14:53:22.006 UTC [27223] LOG: background worker "logical replication launcher" (PID 27232) exited with exit code 1
2026-09-28 14:53:22.007 UTC [27227] LOG: shutting down
2026-09-28 14:53:22.186 UTC [27223] LOG: database system is shut down
2026-09-28T14:53:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51798 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:53:22 dist-test-slave-vk40 krb5kdc[28748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:53:22 dist-test-slave-vk40 krb5kdc[28748](info): set up 2 sockets
Sep 28 14:53:22 dist-test-slave-vk40 krb5kdc[28748](info): commencing operation
krb5kdc: starting...
W20260928 14:53:23.327194 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:53:23 dist-test-slave-vk40 krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:53:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:53:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:53:23.552543 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46017
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:42021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:46017,127.14.241.253:44055,127.14.241.252:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:23.879156 28761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:23.879487 28761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:23.879613 28761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:23.886937 28761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:53:23.887069 28761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:23.887146 28761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:23.887223 28761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:53:23.887284 28761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:53:23.897312 28761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:42021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:46017,127.14.241.253:44055,127.14.241.252:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:46017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:23.898690 28761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:23.900151 28761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:23.912076 28769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:23.912335 28766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:23.912645 28767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:23.913017 28761 server_base.cc:1034] running on GCE node
I20260928 14:53:23.913879 28761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:23.915481 28761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:23.916707 28761 hybrid_clock.cc:648] HybridClock initialized: now 1790607203916641 us; error 61 us; skew 500 ppm
I20260928 14:53:23.917086 28761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:23 dist-test-slave-vk40 krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607203, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:23.920275 28761 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:53:23.921820 28761 webserver.cc:460] Webserver started at http://127.14.241.254:40487/ using document root <none> and password file <none>
I20260928 14:53:23.922556 28761 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:23.922662 28761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:23.922928 28761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:23.925961 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "24630f02564b4d0f8897726a7d6b3691"
format_stamp: "Formatted at 2026-09-28 14:53:23 on dist-test-slave-vk40"
server_key: "f522df48a23ebdbb3ceef8923775c398"
server_key_iv: "6d0c19f198ba548b5ea684364e1be2a2"
server_key_version: "encryptionkey@0"
I20260928 14:53:23.926723 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "24630f02564b4d0f8897726a7d6b3691"
format_stamp: "Formatted at 2026-09-28 14:53:23 on dist-test-slave-vk40"
server_key: "f522df48a23ebdbb3ceef8923775c398"
server_key_iv: "6d0c19f198ba548b5ea684364e1be2a2"
server_key_version: "encryptionkey@0"
I20260928 14:53:23.932101 28761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 14:53:23.935686 28776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:23.937142 28761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:53:23.937307 28761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "24630f02564b4d0f8897726a7d6b3691"
format_stamp: "Formatted at 2026-09-28 14:53:23 on dist-test-slave-vk40"
server_key: "f522df48a23ebdbb3ceef8923775c398"
server_key_iv: "6d0c19f198ba548b5ea684364e1be2a2"
server_key_version: "encryptionkey@0"
I20260928 14:53:23.937501 28761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:23.950274 28761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:23.954711 28761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:23.954993 28761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:23.972833 28761 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:46017
I20260928 14:53:23.972855 28827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:46017 every 8 connection(s)
I20260928 14:53:23.974341 28761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:53:23.978120 28828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:23.984020 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 28761
I20260928 14:53:23.984217 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:53:23.984916 15303 external_mini_cluster.cc:1428] Setting key df08f5628814979116c4d2b81d5fe9b2
I20260928 14:53:23.985126 28828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:53:23 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607203, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.241.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.253@KRBTEST.COM" created.
I20260928 14:53:24.008728 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:23.988589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46017 (local address 127.0.0.1:34144)
0928 14:53:23.989738 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:23.989782 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:23.991683 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:23.992690 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:23.992698 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:23.993448 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:23.993805 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:23.993824 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:23.995418 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:23.995423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:23.996603 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:23.996617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:23.997553 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:23.997560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:23.997835 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:23.998505 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:23.998529 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:24.001009 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:24.004457 (+ 3448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:24.004470 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:24.004485 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:24.004876 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:24.005296 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:24.005302 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:24.005305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:24.005455 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:24.006250 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:24.006258 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:24.006460 (+ 202us) client_negotiation.cc:769] Sending connection context
0928 14:53:24.006686 (+ 226us) client_negotiation.cc:240] Negotiation successful
0928 14:53:24.007282 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":176,"thread_start_us":722,"threads_started":1}
I20260928 14:53:24.008766 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:23.988582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34144 (local address 127.14.241.254:46017)
0928 14:53:23.989455 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:23.989475 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:23.989912 (+ 437us) server_negotiation.cc:407] Connection header received
0928 14:53:23.992116 (+ 2204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:23.992122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:23.992443 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:23.992640 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:23.994019 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:23.995252 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:23.996742 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:23.997313 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:23.997504 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:24.001248 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:24.001256 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:24.001270 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:24.001302 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:24.004321 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:24.005031 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:24.005037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:24.005039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:24.005185 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:24.005568 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:24.005571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:24.005573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:24.006116 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:24.006290 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:24.006858 (+ 568us) server_negotiation.cc:299] Negotiation successful
0928 14:53:24.007138 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":612,"thread_start_us":386,"threads_started":1}
I20260928 14:53:24.012241 28828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:53:24.013388 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.012964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47026)
0928 14:53:24.013146 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.013268 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260928 14:53:24.013670 28778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
I20260928 14:53:24.015292 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.014893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47034)
0928 14:53:24.015078 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.015154 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260928 14:53:24.015519 28828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:44055: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab.
Entry for principal kudu/127.14.241.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab.
I20260928 14:53:24.036511 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.253:44055
--webserver_interface=127.14.241.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab
--principal=kudu/127.14.241.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:42021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:46017,127.14.241.253:44055,127.14.241.252:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:53:24.064893 28828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } attempt: 1
I20260928 14:53:24.066162 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.065658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47046)
0928 14:53:24.065902 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.066030 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260928 14:53:24.067241 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.066855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47048)
0928 14:53:24.067016 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.067095 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260928 14:53:24.067448 28828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:44055: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
I20260928 14:53:24.135717 28828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } attempt: 2
I20260928 14:53:24.137055 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.136552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47060)
0928 14:53:24.136802 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.136911 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260928 14:53:24.138262 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.137826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47074)
0928 14:53:24.137988 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.138079 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260928 14:53:24.138453 28828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:44055: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
I20260928 14:53:24.229796 28828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } attempt: 3
I20260928 14:53:24.231015 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.230500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47086)
0928 14:53:24.230722 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.230840 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260928 14:53:24.232338 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.231878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47102)
0928 14:53:24.232057 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.232136 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260928 14:53:24.232568 28828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:44055: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
W20260928 14:53:24.369913 28835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:24.370254 28835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:24.370400 28835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260928 14:53:24.375846 28828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } attempt: 4
I20260928 14:53:24.377228 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.376601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47114)
0928 14:53:24.376983 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.377082 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229}
W20260928 14:53:24.378356 28835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260928 14:53:24.378377 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.377976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47120)
0928 14:53:24.378138 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.378238 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260928 14:53:24.378473 28835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:24.378566 28835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:24.378585 28828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:44055: Network error: Client connection negotiation failed: client connection to 127.14.241.253:44055: connect: Connection refused (error 111)
W20260928 14:53:24.378647 28835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:53:24.378703 28835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:53:24.389083 28835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:42021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:46017,127.14.241.253:44055,127.14.241.252:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab
--principal=kudu/127.14.241.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.253:44055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.241.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:24.390466 28835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:24.391840 28835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:24.403920 28841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:24.403872 28840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:24.403951 28843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:24.404804 28835 server_base.cc:1034] running on GCE node
I20260928 14:53:24.405704 28835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:24.407320 28835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:24.408560 28835 hybrid_clock.cc:648] HybridClock initialized: now 1790607204408501 us; error 48 us; skew 500 ppm
I20260928 14:53:24.408959 28835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:24 dist-test-slave-vk40 krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607204, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:24.412256 28835 init.cc:377] Logged in from keytab as kudu/127.14.241.253@KRBTEST.COM (short username kudu)
I20260928 14:53:24.413969 28835 webserver.cc:460] Webserver started at http://127.14.241.253:40169/ using document root <none> and password file <none>
I20260928 14:53:24.414698 28835 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:24.414870 28835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:24.415179 28835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:24.418391 28835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/data/instance:
uuid: "e53d860214bb476e8f7bf1427122bf31"
format_stamp: "Formatted at 2026-09-28 14:53:24 on dist-test-slave-vk40"
server_key: "ff4dad418ea000ce800812532ae1c3ea"
server_key_iv: "68aa7d51d0bb4731070e1fce3b3422c4"
server_key_version: "encryptionkey@0"
I20260928 14:53:24.419196 28835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/wal/instance:
uuid: "e53d860214bb476e8f7bf1427122bf31"
format_stamp: "Formatted at 2026-09-28 14:53:24 on dist-test-slave-vk40"
server_key: "ff4dad418ea000ce800812532ae1c3ea"
server_key_iv: "68aa7d51d0bb4731070e1fce3b3422c4"
server_key_version: "encryptionkey@0"
I20260928 14:53:24.424629 28835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260928 14:53:24.428222 28850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:24.429688 28835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:53:24.429873 28835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/wal
uuid: "e53d860214bb476e8f7bf1427122bf31"
format_stamp: "Formatted at 2026-09-28 14:53:24 on dist-test-slave-vk40"
server_key: "ff4dad418ea000ce800812532ae1c3ea"
server_key_iv: "68aa7d51d0bb4731070e1fce3b3422c4"
server_key_version: "encryptionkey@0"
I20260928 14:53:24.430071 28835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:24.450845 28835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:24.455279 28835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:24.455562 28835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:24.474030 28835 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.253:44055
I20260928 14:53:24.474080 28901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.253:44055 every 8 connection(s)
I20260928 14:53:24.475421 28835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
I20260928 14:53:24.479163 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 28835
I20260928 14:53:24.479368 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-1/wal/instance
I20260928 14:53:24.479425 28902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:24.479820 15303 external_mini_cluster.cc:1428] Setting key d567876ba48a2ae4aa22387900cbe9c0
I20260928 14:53:24.486258 28902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.241.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.252@KRBTEST.COM" created.
Sep 28 14:53:24 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607204, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:24.509004 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.489612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.14.241.254:46017)
0928 14:53:24.489840 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:24.489847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:24.491096 (+ 1249us) server_negotiation.cc:407] Connection header received
0928 14:53:24.493143 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:24.493149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:24.493232 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:24.493354 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:24.494939 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.496004 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.497717 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.498390 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.498556 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:24.503142 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:24.503147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:24.503151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:24.503189 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:24.505795 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:24.506581 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:24.506585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:24.506587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:24.506727 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:24.507163 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:24.507167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:24.507169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:24.507486 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:24.507626 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:24.508454 (+ 828us) server_negotiation.cc:299] Negotiation successful
0928 14:53:24.508696 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 14:53:24.509969 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.490044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46017 (local address 127.0.0.1:34148)
0928 14:53:24.490892 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.490955 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:24.492856 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:24.493565 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:24.493574 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:24.494124 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:24.494729 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:24.494748 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.496199 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.496203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:24.497544 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:24.497558 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.498611 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.498617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:24.498825 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:24.499544 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:24.499569 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:24.502871 (+ 3302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:24.505958 (+ 3087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:24.505969 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:24.505987 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:24.506440 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:24.506846 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:24.506851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:24.506854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:24.507042 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:24.507623 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:24.507631 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:24.508031 (+ 400us) client_negotiation.cc:769] Sending connection context
0928 14:53:24.508296 (+ 265us) client_negotiation.cc:240] Negotiation successful
0928 14:53:24.508834 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":357,"threads_started":1}
I20260928 14:53:24.511833 28902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:53:24 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607204, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
I20260928 14:53:24.527122 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.512687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47132)
0928 14:53:24.512863 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.512882 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:24.513014 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:24.513835 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:24.513838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:24.514218 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:24.514368 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:24.514377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.516136 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.516140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:24.517028 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:24.517040 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.518204 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.518210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:24.518370 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:24.518988 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:24.519006 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:24.521033 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:24.524795 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:24.524799 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:24.524802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:24.525164 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:24.525533 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:24.525536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:24.525538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:24.525666 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:24.526328 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:24.526331 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:24.526447 (+ 116us) client_negotiation.cc:769] Sending connection context
0928 14:53:24.526557 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 14:53:24.526808 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260928 14:53:24.527364 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.512719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.14.241.253:44055)
0928 14:53:24.513341 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:24.513349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:24.513389 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:53:24.513444 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:24.513451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:24.513688 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:24.513829 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:24.514532 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.516002 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.517164 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.517965 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.518086 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:24.521281 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:24.521308 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:24.521328 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:24.521368 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:24.524630 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:24.525287 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:24.525295 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:24.525311 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:24.525436 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:24.525765 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:24.525769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:24.525772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:24.526222 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:24.526385 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:24.526722 (+ 337us) server_negotiation.cc:299] Negotiation successful
0928 14:53:24.526935 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":294,"threads_started":1}
I20260928 14:53:24.529767 28902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab.
Entry for principal kudu/127.14.241.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab.
I20260928 14:53:24.531021 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.530478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59414)
0928 14:53:24.530668 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.530810 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260928 14:53:24.531340 28854 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
I20260928 14:53:24.531924 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.252:40807
--webserver_interface=127.14.241.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab
--principal=kudu/127.14.241.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:42021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:46017,127.14.241.253:44055,127.14.241.252:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:53:24.532995 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.532622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59418)
0928 14:53:24.532804 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.532887 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260928 14:53:24.533226 28902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:40807: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
I20260928 14:53:24.582578 28902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } attempt: 1
I20260928 14:53:24.583776 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.583313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59426)
0928 14:53:24.583509 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.583619 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260928 14:53:24.584925 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.584556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59430)
0928 14:53:24.584729 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.584805 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260928 14:53:24.585147 28902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:40807: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
I20260928 14:53:24.653347 28902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } attempt: 2
I20260928 14:53:24.654537 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.654092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59434)
0928 14:53:24.654291 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.654383 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260928 14:53:24.655545 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.655200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59436)
0928 14:53:24.655355 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.655427 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260928 14:53:24.655761 28902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:40807: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
I20260928 14:53:24.677860 28828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } attempt: 5
Sep 28 14:53:24 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607203, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
I20260928 14:53:24.691656 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.678629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.14.241.253:44055)
0928 14:53:24.678798 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:24.678802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:24.678907 (+ 105us) server_negotiation.cc:407] Connection header received
0928 14:53:24.679095 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:24.679099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:24.679175 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:24.679257 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:24.680324 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.681018 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.682691 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.683124 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.683215 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:24.686722 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:24.686726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:24.686730 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:24.686762 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:24.689216 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:24.689841 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:24.689847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:24.689850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:24.689968 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:24.690325 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:24.690328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:24.690330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:24.690614 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:24.690743 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:24.691138 (+ 395us) server_negotiation.cc:299] Negotiation successful
0928 14:53:24.691366 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 14:53:24.691659 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.678525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47134)
0928 14:53:24.678806 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.678829 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:24.678983 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:24.679340 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:24.679345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:24.679866 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:24.680141 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:24.680155 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.681159 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.681166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:24.682558 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:24.682574 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.683279 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.683285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:24.683470 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:24.684098 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:24.684122 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:24.686513 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:24.689358 (+ 2845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:24.689363 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:24.689368 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:24.689724 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:24.690078 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:24.690083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:24.690086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:24.690221 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:24.690731 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:24.690735 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:24.690892 (+ 157us) client_negotiation.cc:769] Sending connection context
0928 14:53:24.691015 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 14:53:24.691300 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260928 14:53:24.692749 28828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:53:24.693964 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.693580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59446)
0928 14:53:24.693748 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.693826 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 14:53:24.694854 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.694526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59452)
0928 14:53:24.694665 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.694742 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 14:53:24.695037 28828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:40807: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
I20260928 14:53:24.746253 28828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } attempt: 1
I20260928 14:53:24.746951 28902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } attempt: 3
I20260928 14:53:24.747498 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.746955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59468)
0928 14:53:24.747195 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.747317 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260928 14:53:24.748173 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.747585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59478)
0928 14:53:24.747885 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.747986 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260928 14:53:24.748508 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.748144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59482)
0928 14:53:24.748312 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.748392 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260928 14:53:24.748792 28828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:40807: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
I20260928 14:53:24.749410 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.748947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59484)
0928 14:53:24.749206 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.749290 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
W20260928 14:53:24.749629 28902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:40807: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
I20260928 14:53:24.817008 28828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } attempt: 2
I20260928 14:53:24.818156 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.817688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59498)
0928 14:53:24.817904 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.817996 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260928 14:53:24.819121 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.818742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59504)
0928 14:53:24.818892 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.818999 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260928 14:53:24.819319 28828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:40807: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
W20260928 14:53:24.860898 28909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:24.861209 28909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:24.861343 28909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:24.868639 28909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:53:24.868773 28909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:24.868860 28909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:24.868934 28909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:53:24.868994 28909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:53:24.878868 28909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:42021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:46017,127.14.241.253:44055,127.14.241.252:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab
--principal=kudu/127.14.241.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.252:40807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.241.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:24.880205 28909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:24.881526 28909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:53:24.892820 28902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } attempt: 4
W20260928 14:53:24.892999 28915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:24.893046 28917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:24.893559 28914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:24.894098 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.893433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59506)
0928 14:53:24.893845 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.893935 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229}
I20260928 14:53:24.894304 28909 server_base.cc:1034] running on GCE node
I20260928 14:53:24.895066 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.894746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59522)
0928 14:53:24.894887 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.894955 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 14:53:24.895074 28909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260928 14:53:24.895267 28902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:40807: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
I20260928 14:53:24.896572 28909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:24.897816 28909 hybrid_clock.cc:648] HybridClock initialized: now 1790607204897754 us; error 51 us; skew 500 ppm
I20260928 14:53:24.898190 28909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:24 dist-test-slave-vk40 krb5kdc[28748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607204, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:24.901257 28909 init.cc:377] Logged in from keytab as kudu/127.14.241.252@KRBTEST.COM (short username kudu)
I20260928 14:53:24.902693 28909 webserver.cc:460] Webserver started at http://127.14.241.252:40151/ using document root <none> and password file <none>
I20260928 14:53:24.903375 28909 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:24.903491 28909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:24.903772 28909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:24.906857 28909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/data/instance:
uuid: "5c1f1d41a9874cdfaacdd7c40494b95a"
format_stamp: "Formatted at 2026-09-28 14:53:24 on dist-test-slave-vk40"
server_key: "fb2157213ec465dd3818c04b7253abbc"
server_key_iv: "b1b41a57e3d9c1eae87ff4173ee4a1b1"
server_key_version: "encryptionkey@0"
I20260928 14:53:24.907651 28909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/wal/instance:
uuid: "5c1f1d41a9874cdfaacdd7c40494b95a"
format_stamp: "Formatted at 2026-09-28 14:53:24 on dist-test-slave-vk40"
server_key: "fb2157213ec465dd3818c04b7253abbc"
server_key_iv: "b1b41a57e3d9c1eae87ff4173ee4a1b1"
server_key_version: "encryptionkey@0"
I20260928 14:53:24.912853 28909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 14:53:24.916370 28924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:24.917711 28909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:53:24.917900 28909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/wal
uuid: "5c1f1d41a9874cdfaacdd7c40494b95a"
format_stamp: "Formatted at 2026-09-28 14:53:24 on dist-test-slave-vk40"
server_key: "fb2157213ec465dd3818c04b7253abbc"
server_key_iv: "b1b41a57e3d9c1eae87ff4173ee4a1b1"
server_key_version: "encryptionkey@0"
I20260928 14:53:24.918083 28909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:24.925525 28828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } attempt: 3
I20260928 14:53:24.926649 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.926140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59528)
0928 14:53:24.926371 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.926474 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 14:53:24.927691 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.927263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59530)
0928 14:53:24.927484 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.927568 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260928 14:53:24.927929 28828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:40807: Network error: Client connection negotiation failed: client connection to 127.14.241.252:40807: connect: Connection refused (error 111)
I20260928 14:53:24.947902 28909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:24.952126 28909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:24.952421 28909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:24.970089 28909 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.252:40807
I20260928 14:53:24.970125 28975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.252:40807 every 8 connection(s)
I20260928 14:53:24.971407 28909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
I20260928 14:53:24.974118 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 28909
I20260928 14:53:24.974326 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790607004819300-15303-0/minicluster-data/master-2/wal/instance
I20260928 14:53:24.974874 15303 external_mini_cluster.cc:1428] Setting key d10b7d0b14ee4ff71232ea6158798196
I20260928 14:53:24.975644 28976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:24.985844 28976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:53:24 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
Sep 28 14:53:24 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607204, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:24.999105 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.976795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59534 (local address 127.14.241.252:40807)
0928 14:53:24.977579 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:24.977595 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:24.977646 (+ 51us) server_negotiation.cc:407] Connection header received
0928 14:53:24.979840 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:24.979880 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:24.980454 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:24.980879 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:24.982750 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.984130 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.985986 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.986761 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.987042 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:24.991871 (+ 4829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:24.991879 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:24.991890 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:24.991926 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:24.994875 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:24.995580 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:24.995587 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:24.995593 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:24.995767 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:24.996145 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:24.996148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:24.996150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:24.996896 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:24.997178 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:24.997632 (+ 454us) server_negotiation.cc:299] Negotiation successful
0928 14:53:24.997980 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":1119,"threads_started":1}
I20260928 14:53:25.002744 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.988671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:46017 (local address 127.0.0.1:34162)
0928 14:53:24.989429 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:24.989471 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:24.989635 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:24.990003 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:24.990010 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:24.990773 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:24.991099 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:24.991109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.992100 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.992104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:24.993251 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:24.993276 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.994022 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.994028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:24.994185 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:24.994803 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:24.994820 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:24.996990 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:25.000157 (+ 3167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:25.000168 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:25.000174 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:25.000638 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:25.001029 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:25.001032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:25.001034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:25.001153 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:25.001695 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:25.001701 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:25.001863 (+ 162us) client_negotiation.cc:769] Sending connection context
0928 14:53:25.002017 (+ 154us) client_negotiation.cc:240] Negotiation successful
0928 14:53:25.002320 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":416,"threads_started":1}
I20260928 14:53:25.002784 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:24.988588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.14.241.254:46017)
0928 14:53:24.988768 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:24.988773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:24.989575 (+ 802us) server_negotiation.cc:407] Connection header received
0928 14:53:24.989760 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:24.989766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:24.989836 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:24.989921 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:24.991260 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.991937 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.993403 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:24.993828 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:24.993935 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:24.997208 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:24.997215 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:24.997220 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:24.997262 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:25.000025 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.000763 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.000768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.000771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.000922 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.001271 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.001275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.001277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.001573 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:25.001728 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:25.002178 (+ 450us) server_negotiation.cc:299] Negotiation successful
0928 14:53:25.002430 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:53:25.004451 28976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:53:25 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
Sep 28 14:53:25 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607204, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
I20260928 14:53:25.017764 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.005084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:44055 (local address 127.0.0.1:47148)
0928 14:53:25.005256 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:25.005273 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:25.005365 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:25.005830 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:25.005833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:25.006219 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:25.006389 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:25.006400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.007349 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.007355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:25.008496 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:25.008512 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.009283 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.009290 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:25.009518 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.010028 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:25.010051 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:25.012495 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:25.015346 (+ 2851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:25.015351 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:25.015355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:25.015719 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:25.016112 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:25.016118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:25.016122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:25.016268 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:25.016815 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:25.016819 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:25.017006 (+ 187us) client_negotiation.cc:769] Sending connection context
0928 14:53:25.017145 (+ 139us) client_negotiation.cc:240] Negotiation successful
0928 14:53:25.017418 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 14:53:25.017791 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.002187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34164 (local address 127.14.241.254:46017)
0928 14:53:25.003072 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:25.003077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:25.003092 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:53:25.003136 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:25.003141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:25.003205 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:25.003295 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:25.004932 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.005974 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.007419 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.007825 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.007925 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.012521 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:25.012528 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:25.012532 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:25.012572 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:25.015211 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.015916 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.015921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.015924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.016049 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.016476 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.016480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.016482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.016770 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:25.017042 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:25.017245 (+ 203us) server_negotiation.cc:299] Negotiation successful
0928 14:53:25.017453 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":793,"thread_start_us":2210,"threads_started":1}
I20260928 14:53:25.018040 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.005273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.14.241.253:44055)
0928 14:53:25.005483 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:25.005489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:25.005508 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:53:25.005562 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:25.005567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:25.005637 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:25.005740 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:25.006547 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.007213 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.008637 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.009124 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.009218 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.012715 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:25.012731 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:25.012734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:25.012767 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:25.015202 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.015837 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.015843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.015846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.016002 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.016403 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.016407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.016410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.016691 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:25.016809 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:25.017281 (+ 472us) server_negotiation.cc:299] Negotiation successful
0928 14:53:25.017490 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 14:53:25.019050 28976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:53:25 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607204, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
Sep 28 14:53:25 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
I20260928 14:53:25.032644 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.019862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59548)
0928 14:53:25.020024 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:25.020041 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:25.020127 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:25.020468 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:25.020473 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:25.020926 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:25.021106 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:25.021118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.022420 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.022424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:25.023356 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:25.023368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.024054 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.024060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:25.024218 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.024766 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:25.024788 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:25.026979 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:25.030007 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:25.030012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:25.030015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:25.030460 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:25.030844 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:25.030850 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:25.030862 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:25.031020 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:25.031611 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:25.031615 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:25.031750 (+ 135us) client_negotiation.cc:769] Sending connection context
0928 14:53:25.031927 (+ 177us) client_negotiation.cc:240] Negotiation successful
0928 14:53:25.032199 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260928 14:53:25.032687 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.020088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59548 (local address 127.14.241.252:40807)
0928 14:53:25.020219 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:25.020224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:25.020237 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:53:25.020272 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:25.020275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:25.020352 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:25.020421 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:25.021229 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.022285 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.023483 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.023903 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.024006 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.027174 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:25.027181 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:25.027186 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:25.027224 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:25.029876 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.030574 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.030578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.030580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.030709 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.031145 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.031149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.031152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.031496 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:25.031642 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:25.032033 (+ 391us) server_negotiation.cc:299] Negotiation successful
0928 14:53:25.032427 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":46}
I20260928 14:53:25.034183 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.019397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47152 (local address 127.14.241.253:44055)
0928 14:53:25.019545 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:25.019550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:25.019587 (+ 37us) server_negotiation.cc:407] Connection header received
0928 14:53:25.019765 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:25.019770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:25.019838 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:25.019923 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:25.021526 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.022577 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.023777 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.024194 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.024331 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.028371 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:25.028376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:25.028379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:25.028408 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:25.031028 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.031762 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.031767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.031771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.031949 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.032375 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.032381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.032384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.032708 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:25.032986 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:25.033742 (+ 756us) server_negotiation.cc:299] Negotiation successful
0928 14:53:25.033917 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:53:25.038270 28976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a: Bootstrap starting.
I20260928 14:53:25.039019 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:53:25.041672 28976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:25.042930 28976 log.cc:824] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:25.054126 28976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a: No bootstrap required, opened a new log
I20260928 14:53:25.062290 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.043769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34178 (local address 127.14.241.254:46017)
0928 14:53:25.043957 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:25.043962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:25.044779 (+ 817us) server_negotiation.cc:407] Connection header received
0928 14:53:25.045059 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:25.045066 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:25.045157 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:25.045231 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:25.047812 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.049087 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.051234 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.051864 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.051966 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.055812 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:25.055819 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:25.055823 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:25.055867 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:25.059093 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.060077 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.060083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.060086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.060264 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.060711 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.060716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.060719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.061078 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:25.061342 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:25.061665 (+ 323us) server_negotiation.cc:299] Negotiation successful
0928 14:53:25.061909 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:53:25.063776 28976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } }
I20260928 14:53:25.064250 28976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:25.064428 28976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1f1d41a9874cdfaacdd7c40494b95a, State: Initialized, Role: FOLLOWER
I20260928 14:53:25.066298 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.045029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.14.241.253:44055)
0928 14:53:25.046693 (+ 1664us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:25.046701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:25.050469 (+ 3768us) server_negotiation.cc:407] Connection header received
0928 14:53:25.050927 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:25.050934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:25.051017 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:25.051082 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:25.053348 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.054403 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.056508 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.057128 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.057259 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.060071 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:25.060078 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:25.060083 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:25.060127 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:25.063239 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.064130 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.064135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.064137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.064320 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.064754 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.064759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.064763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.065054 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:25.065227 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:25.065716 (+ 489us) server_negotiation.cc:299] Negotiation successful
0928 14:53:25.065956 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1499}
I20260928 14:53:25.066560 28976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } }
I20260928 14:53:25.067680 28990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } } }
I20260928 14:53:25.068061 28990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:53:25.069022 28976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [sys.catalog]: configured and running, proceeding with master startup.
W20260928 14:53:25.079710 29001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:53:25.079931 29001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:53:25.080149 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.045219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.14.241.252:40807)
0928 14:53:25.045407 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:25.045413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:25.055333 (+ 9920us) server_negotiation.cc:407] Connection header received
0928 14:53:25.055514 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:25.055520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:25.055591 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:25.055678 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:25.066137 (+ 10459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.067273 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.069081 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.069599 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.069709 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.072982 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:25.072990 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:25.072995 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:25.073048 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:25.076154 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.077020 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.077026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.077029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.077226 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.077701 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.077707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.077710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.078129 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:25.078424 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:25.079525 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0928 14:53:25.079772 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:53:25.076611 28909 master_runner.cc:186] Error getting master registration for 127.14.241.254:46017: OK, instance_id { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" instance_seqno: 1790607203967511 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:53:25.088081 28909 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:53:25.105146 28828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } attempt: 4
Sep 28 14:53:25 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607203, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
I20260928 14:53:25.121279 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.105696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59558)
0928 14:53:25.105906 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:25.105926 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:25.106052 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:25.106394 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:25.106399 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:25.106817 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:25.107006 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:25.107015 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.108063 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.108067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:25.109208 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:25.109221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.109954 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.109960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:25.110127 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.110682 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:25.110707 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:25.114688 (+ 3981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:25.118917 (+ 4229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:25.118921 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:25.118925 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:25.119333 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:25.119756 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:25.119760 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:25.119762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:25.119900 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:25.120537 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:25.120541 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:25.120671 (+ 130us) client_negotiation.cc:769] Sending connection context
0928 14:53:25.120792 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 14:53:25.121016 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260928 14:53:25.121563 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.105750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59558 (local address 127.14.241.252:40807)
0928 14:53:25.105920 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:25.105927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:25.105995 (+ 68us) server_negotiation.cc:407] Connection header received
0928 14:53:25.106163 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:25.106169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:25.106253 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:25.106342 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:25.107151 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.107931 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.109349 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.109814 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.109927 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.114934 (+ 5007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:25.114941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:25.114946 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:25.114986 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:25.118726 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.119473 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.119479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.119483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.119652 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.120012 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.120017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.120020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.120427 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:25.120588 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:25.120948 (+ 360us) server_negotiation.cc:299] Negotiation successful
0928 14:53:25.121189 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 14:53:25.128398 28828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691: Bootstrap starting.
I20260928 14:53:25.132638 28828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:25.134081 28828 log.cc:824] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:25.137570 28828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691: No bootstrap required, opened a new log
I20260928 14:53:25.148969 28828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } }
I20260928 14:53:25.149492 28828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:25.149628 28828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24630f02564b4d0f8897726a7d6b3691, State: Initialized, Role: FOLLOWER
I20260928 14:53:25.150432 28828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } }
I20260928 14:53:25.151355 29004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } } }
I20260928 14:53:25.151674 29004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:53:25.152849 28828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 14:53:25.163964 29015 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:53:25.164196 29015 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:53:25.161991 28761 master_runner.cc:186] Error getting master registration for 127.14.241.253:44055: OK, instance_id { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" instance_seqno: 1790607204468681 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:53:25.173789 28761 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:53:25.194520 28902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } attempt: 5
Sep 28 14:53:25 dist-test-slave-vk40 krb5kdc[28748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607204, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
W20260928 14:53:25.204619 28930 tablet.cc:2369] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:53:25.208113 28782 tablet.cc:2369] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:53:25.212886 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.195313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:40807 (local address 127.0.0.1:59562)
0928 14:53:25.195616 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:25.195638 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:25.195760 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:25.196185 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:25.196190 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:25.196765 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:25.197008 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:25.197021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.198839 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.198856 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:25.200329 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:25.200347 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.201454 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.201462 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:25.201709 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.202430 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:25.202455 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:25.205466 (+ 3011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:25.209913 (+ 4447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:25.209920 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:25.209924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:25.210457 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:25.210929 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:25.210935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:25.210939 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:25.211136 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:25.211810 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:25.211815 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:25.211987 (+ 172us) client_negotiation.cc:769] Sending connection context
0928 14:53:25.212135 (+ 148us) client_negotiation.cc:240] Negotiation successful
0928 14:53:25.212476 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173}
I20260928 14:53:25.212957 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:25.195518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59562 (local address 127.14.241.252:40807)
0928 14:53:25.195711 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:25.195718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:25.195740 (+ 22us) server_negotiation.cc:407] Connection header received
0928 14:53:25.195917 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:25.195923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:25.196011 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:25.196125 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:25.197210 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.198371 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.200516 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:25.201272 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:25.201414 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:25.205677 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:25.205684 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:25.205689 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:25.205738 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:25.209719 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.210629 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.210635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.210639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.210796 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:25.211267 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:25.211271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:25.211275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:25.211676 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:25.211854 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:25.212270 (+ 416us) server_negotiation.cc:299] Negotiation successful
0928 14:53:25.212556 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:53:25.219472 28902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31: Bootstrap starting.
I20260928 14:53:25.224488 28902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:25.225857 28902 log.cc:824] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:25.229282 28902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31: No bootstrap required, opened a new log
I20260928 14:53:25.241523 28902 raft_consensus.cc:348] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } }
I20260928 14:53:25.241984 28902 raft_consensus.cc:374] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:25.242101 28902 raft_consensus.cc:729] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e53d860214bb476e8f7bf1427122bf31, State: Initialized, Role: FOLLOWER
I20260928 14:53:25.242815 28902 consensus_queue.cc:260] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } }
I20260928 14:53:25.243831 29018 sys_catalog.cc:455] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } } }
I20260928 14:53:25.244328 29018 sys_catalog.cc:458] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:53:25.245239 28902 sys_catalog.cc:564] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:53:25.262606 28835 master_runner.cc:263] No leader master found from master e53d860214bb476e8f7bf1427122bf31
I20260928 14:53:25.262764 28835 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260928 14:53:25.271443 29029 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:53:25.271620 29029 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260928 14:53:25.458388 28856 tablet.cc:2369] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:53:25.480504 28990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:53:25.480850 28990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } }
I20260928 14:53:25.482558 28990 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24630f02564b4d0f8897726a7d6b3691 (127.14.241.254:46017), e53d860214bb476e8f7bf1427122bf31 (127.14.241.253:44055)
I20260928 14:53:25.483793 28803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24630f02564b4d0f8897726a7d6b3691" is_pre_election: true
I20260928 14:53:25.484454 28803 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c1f1d41a9874cdfaacdd7c40494b95a in term 0.
I20260928 14:53:25.485500 28927 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24630f02564b4d0f8897726a7d6b3691, 5c1f1d41a9874cdfaacdd7c40494b95a; no voters:
I20260928 14:53:25.485492 28877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53d860214bb476e8f7bf1427122bf31" is_pre_election: true
I20260928 14:53:25.485946 28990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:53:25.486089 28990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:53:25.486073 28877 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c1f1d41a9874cdfaacdd7c40494b95a in term 0.
I20260928 14:53:25.486172 28990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:25.489389 28990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } }
I20260928 14:53:25.490499 28803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24630f02564b4d0f8897726a7d6b3691"
I20260928 14:53:25.490782 28803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:25.493871 28803 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c1f1d41a9874cdfaacdd7c40494b95a in term 1.
I20260928 14:53:25.494304 28927 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24630f02564b4d0f8897726a7d6b3691, 5c1f1d41a9874cdfaacdd7c40494b95a; no voters:
I20260928 14:53:25.496932 28877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53d860214bb476e8f7bf1427122bf31"
I20260928 14:53:25.497085 29031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:53:25.497175 28877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:25.500594 28877 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c1f1d41a9874cdfaacdd7c40494b95a in term 1.
I20260928 14:53:25.501379 28990 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [CANDIDATE]: Term 1 election: Requested vote from peers 24630f02564b4d0f8897726a7d6b3691 (127.14.241.254:46017), e53d860214bb476e8f7bf1427122bf31 (127.14.241.253:44055)
I20260928 14:53:25.503839 29031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [term 1 LEADER]: Becoming Leader. State: Replica: 5c1f1d41a9874cdfaacdd7c40494b95a, State: Running, Role: LEADER
I20260928 14:53:25.504526 29031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } }
I20260928 14:53:25.516386 28990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1f1d41a9874cdfaacdd7c40494b95a. Latest consensus state: current_term: 1 leader_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } } }
I20260928 14:53:25.516755 28990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:25.528908 29034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:53:25.531105 29034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:53:25.539759 28877 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [term 1 FOLLOWER]: Refusing update from remote peer 5c1f1d41a9874cdfaacdd7c40494b95a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:53:25.540007 28803 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [term 1 FOLLOWER]: Refusing update from remote peer 5c1f1d41a9874cdfaacdd7c40494b95a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:53:25.540928 28990 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:53:25.541263 29031 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [LEADER]: Connected to new peer: Peer: permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:53:25.563663 29004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1f1d41a9874cdfaacdd7c40494b95a. Latest consensus state: current_term: 1 leader_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } } }
I20260928 14:53:25.563997 29004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:53:25.568058 29035 mvcc.cc:205] Tried to move back new op lower bound from 7334327113880588288 to 7334327113752489984. Current Snapshot: MvccSnapshot[applied={T|T < 7334327113880588288}]
I20260928 14:53:25.573025 29037 mvcc.cc:205] Tried to move back new op lower bound from 7334327113880588288 to 7334327113752489984. Current Snapshot: MvccSnapshot[applied={T|T < 7334327113880588288}]
I20260928 14:53:25.574842 29018 sys_catalog.cc:455] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1f1d41a9874cdfaacdd7c40494b95a. Latest consensus state: current_term: 1 leader_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } } }
I20260928 14:53:25.575209 29018 sys_catalog.cc:458] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:53:25.581208 29018 sys_catalog.cc:455] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } } }
I20260928 14:53:25.581465 29018 sys_catalog.cc:458] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:53:25.583854 29031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } } }
I20260928 14:53:25.586431 29004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } } }
I20260928 14:53:25.586669 29004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:53:25.589118 29034 catalog_manager.cc:1347] Generated new cluster ID: c3c9e267aa5740c0b9a0b353b67afbdb
I20260928 14:53:25.589205 29034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:53:25.592847 29031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:25.593609 29031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24630f02564b4d0f8897726a7d6b3691" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 46017 } } peers { permanent_uuid: "e53d860214bb476e8f7bf1427122bf31" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 44055 } } peers { permanent_uuid: "5c1f1d41a9874cdfaacdd7c40494b95a" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 40807 } } }
I20260928 14:53:25.593863 29031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:25.606544 29034 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:53:25.608626 29034 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:53:25.620631 29034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c1f1d41a9874cdfaacdd7c40494b95a: Generated new TSK 0
I20260928 14:53:25.621912 29034 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:53:26.166167 29015 catalog_manager.cc:1259] Loaded cluster ID: c3c9e267aa5740c0b9a0b353b67afbdb
I20260928 14:53:26.166265 29015 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691: loading cluster ID for follower catalog manager: success
I20260928 14:53:26.169201 29015 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691: acquiring CA information for follower catalog manager: success
I20260928 14:53:26.171003 29015 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 24630f02564b4d0f8897726a7d6b3691: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:53:26.273628 29029 catalog_manager.cc:1259] Loaded cluster ID: c3c9e267aa5740c0b9a0b353b67afbdb
I20260928 14:53:26.273739 29029 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31: loading cluster ID for follower catalog manager: success
I20260928 14:53:26.276146 29029 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31: acquiring CA information for follower catalog manager: success
I20260928 14:53:26.277931 29029 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e53d860214bb476e8f7bf1427122bf31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:53:26.715806 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 28761
I20260928 14:53:26.717126 28823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:26.828539 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 28761
I20260928 14:53:26.846292 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 28835
I20260928 14:53:26.847412 28897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:26.954082 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 28835
I20260928 14:53:26.971799 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 28909
I20260928 14:53:26.972895 28971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:27.082022 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 28909
2026-09-28T14:53:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4783 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:53:27 dist-test-slave-vk40 krb5kdc[29051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:53:27 dist-test-slave-vk40 krb5kdc[29051](info): set up 2 sockets
Sep 28 14:53:27 dist-test-slave-vk40 krb5kdc[29051](info): commencing operation
krb5kdc: starting...
W20260928 14:53:28.115396 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:53:28 dist-test-slave-vk40 krb5kdc[29051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:53:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:53:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:53:28.347627 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38541
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:46011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:38541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:28.675525 29064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:28.675836 29064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:28.675980 29064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:28.683195 29064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:53:28.683319 29064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:28.683401 29064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:28.683480 29064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:53:28.683542 29064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:53:28.693531 29064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:46011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:38541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:28.694823 29064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:28.696125 29064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:28.707911 29069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:28.707876 29072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:28.707937 29070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:28.709141 29064 server_base.cc:1034] running on GCE node
I20260928 14:53:28.709923 29064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:28.711329 29064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:28.712538 29064 hybrid_clock.cc:648] HybridClock initialized: now 1790607208712483 us; error 42 us; skew 500 ppm
I20260928 14:53:28.712898 29064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:28 dist-test-slave-vk40 krb5kdc[29051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607208, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:28.721983 29064 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:53:28.723714 29064 webserver.cc:460] Webserver started at http://127.14.241.254:44133/ using document root <none> and password file <none>
I20260928 14:53:28.724457 29064 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:28.724581 29064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:28.724840 29064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:28.727865 29064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "1c872a1f03194def867a2858aac06991"
format_stamp: "Formatted at 2026-09-28 14:53:28 on dist-test-slave-vk40"
server_key: "dc79d4310b82a60ad357bb649f5f7dca"
server_key_iv: "78ed9b0db6c53635cf3757350800be6e"
server_key_version: "encryptionkey@0"
I20260928 14:53:28.728621 29064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "1c872a1f03194def867a2858aac06991"
format_stamp: "Formatted at 2026-09-28 14:53:28 on dist-test-slave-vk40"
server_key: "dc79d4310b82a60ad357bb649f5f7dca"
server_key_iv: "78ed9b0db6c53635cf3757350800be6e"
server_key_version: "encryptionkey@0"
I20260928 14:53:28.733870 29064 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:53:28.737336 29079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:28.738689 29064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 14:53:28.738852 29064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "1c872a1f03194def867a2858aac06991"
format_stamp: "Formatted at 2026-09-28 14:53:28 on dist-test-slave-vk40"
server_key: "dc79d4310b82a60ad357bb649f5f7dca"
server_key_iv: "78ed9b0db6c53635cf3757350800be6e"
server_key_version: "encryptionkey@0"
I20260928 14:53:28.739043 29064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:28.770396 29064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:28.774708 29064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:28.774983 29064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:28.792502 29064 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:38541
I20260928 14:53:28.792501 29130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:38541 every 8 connection(s)
I20260928 14:53:28.793866 29064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:53:28.797665 29131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:28.800319 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 29064
I20260928 14:53:28.800550 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:53:28.801384 15303 external_mini_cluster.cc:1428] Setting key f653fe1b21a88c20f97d914eb57557e0
I20260928 14:53:28.810115 29131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991: Bootstrap starting.
I20260928 14:53:28.813913 29131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:28.815178 29131 log.cc:824] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991: Log is configured to *not* fsync() on all Append() calls
Sep 28 14:53:28 dist-test-slave-vk40 krb5kdc[29051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:28.818320 29131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991: No bootstrap required, opened a new log
I20260928 14:53:28.826252 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:28.803330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37420 (local address 127.14.241.254:38541)
0928 14:53:28.804152 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:28.804164 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:28.804204 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:53:28.805923 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:28.805947 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:28.806370 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:28.806734 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:28.808587 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:28.809957 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:28.811745 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:28.812500 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:28.812626 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:28.817532 (+ 4906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:28.817542 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:28.817562 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:28.817604 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:28.821591 (+ 3987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:28.822338 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:28.822350 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:28.822359 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:28.822576 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:28.822914 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:28.822919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:28.822923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:28.823679 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:28.823996 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:28.824394 (+ 398us) server_negotiation.cc:299] Negotiation successful
0928 14:53:28.824877 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":369,"threads_started":1}
I20260928 14:53:28.828707 29131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c872a1f03194def867a2858aac06991" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38541 } }
I20260928 14:53:28.829152 29131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:28.829264 29131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c872a1f03194def867a2858aac06991, State: Initialized, Role: FOLLOWER
I20260928 14:53:28.829901 29131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c872a1f03194def867a2858aac06991" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38541 } }
I20260928 14:53:28.830190 29131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:53:28.830376 29131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:53:28.830502 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:53:28.830549 29131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:28.832831 29131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c872a1f03194def867a2858aac06991" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38541 } }
I20260928 14:53:28.833405 29131 leader_election.cc:302] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c872a1f03194def867a2858aac06991; no voters:
I20260928 14:53:28.834022 29131 leader_election.cc:288] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:53:28.834151 29136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:53:28.834791 29136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [term 1 LEADER]: Becoming Leader. State: Replica: 1c872a1f03194def867a2858aac06991, State: Running, Role: LEADER
I20260928 14:53:28.835290 29136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c872a1f03194def867a2858aac06991" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38541 } }
I20260928 14:53:28.835898 29131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:53:28.838114 29137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c872a1f03194def867a2858aac06991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c872a1f03194def867a2858aac06991" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38541 } } }
I20260928 14:53:28.838511 29137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:28.838891 29138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c872a1f03194def867a2858aac06991. Latest consensus state: current_term: 1 leader_uuid: "1c872a1f03194def867a2858aac06991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c872a1f03194def867a2858aac06991" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38541 } } }
I20260928 14:53:28.839180 29138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:28.840633 29145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:53:28.845575 29145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:53:28.854187 29145 catalog_manager.cc:1347] Generated new cluster ID: 0cb56dbe704f4354a79d58e5830946f8
I20260928 14:53:28.854305 29145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:53:28.867015 29145 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:53:28.868909 29145 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:53:28.882092 29145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c872a1f03194def867a2858aac06991: Generated new TSK 0
I20260928 14:53:28.883262 29145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:53:28.942023 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 29064
I20260928 14:53:28.943222 29126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:29.040992 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 29064
2026-09-28T14:53:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1953 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:53:29 dist-test-slave-vk40 krb5kdc[29159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:53:29 dist-test-slave-vk40 krb5kdc[29159](info): set up 2 sockets
Sep 28 14:53:29 dist-test-slave-vk40 krb5kdc[29159](info): commencing operation
krb5kdc: starting...
W20260928 14:53:30.055512 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:53:30 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:53:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:53:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:53:30.289088 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38925
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:38925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:30.613325 29172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:30.613644 29172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:30.613788 29172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:30.621440 29172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:53:30.621572 29172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:30.621654 29172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:30.621742 29172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:53:30.621804 29172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:53:30.632907 29172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:38925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:30.634234 29172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:30.635519 29172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:30.646947 29177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:30.646996 29180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:30.646908 29178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:30.647787 29172 server_base.cc:1034] running on GCE node
I20260928 14:53:30.648583 29172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:30.650034 29172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:30.651253 29172 hybrid_clock.cc:648] HybridClock initialized: now 1790607210651197 us; error 47 us; skew 500 ppm
I20260928 14:53:30.651598 29172 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:30 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607210, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:30.654639 29172 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:53:30.656157 29172 webserver.cc:460] Webserver started at http://127.14.241.254:37827/ using document root <none> and password file <none>
I20260928 14:53:30.656883 29172 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:30.657003 29172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:30.657277 29172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:30.660185 29172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "6344471ac5fd4c6bb2e0e300525ba823"
format_stamp: "Formatted at 2026-09-28 14:53:30 on dist-test-slave-vk40"
server_key: "528c0f03898e3b5322ca8303892e2bc3"
server_key_iv: "63611d91a577b0da8bee6db70f6e51c4"
server_key_version: "encryptionkey@0"
I20260928 14:53:30.660972 29172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "6344471ac5fd4c6bb2e0e300525ba823"
format_stamp: "Formatted at 2026-09-28 14:53:30 on dist-test-slave-vk40"
server_key: "528c0f03898e3b5322ca8303892e2bc3"
server_key_iv: "63611d91a577b0da8bee6db70f6e51c4"
server_key_version: "encryptionkey@0"
I20260928 14:53:30.666033 29172 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:53:30.669454 29187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:30.670924 29172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:53:30.671082 29172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "6344471ac5fd4c6bb2e0e300525ba823"
format_stamp: "Formatted at 2026-09-28 14:53:30 on dist-test-slave-vk40"
server_key: "528c0f03898e3b5322ca8303892e2bc3"
server_key_iv: "63611d91a577b0da8bee6db70f6e51c4"
server_key_version: "encryptionkey@0"
I20260928 14:53:30.671273 29172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:30.702214 29172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:30.706888 29172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:30.707180 29172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:30.724200 29172 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:38925
I20260928 14:53:30.724231 29238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:38925 every 8 connection(s)
I20260928 14:53:30.725598 29172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:53:30.729444 29239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:30.731477 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 29172
I20260928 14:53:30.731691 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:53:30.732544 15303 external_mini_cluster.cc:1428] Setting key 78a62529a3a4117908e0a929a30401e9
I20260928 14:53:30.741850 29239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Bootstrap starting.
I20260928 14:53:30.745575 29239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:30.746877 29239 log.cc:824] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Log is configured to *not* fsync() on all Append() calls
Sep 28 14:53:30 dist-test-slave-vk40 krb5kdc[29159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:30.749819 29239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: No bootstrap required, opened a new log
I20260928 14:53:30.755757 29242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:30.734760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52000 (local address 127.14.241.254:38925)
0928 14:53:30.735557 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:30.735570 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:30.735616 (+ 46us) server_negotiation.cc:407] Connection header received
0928 14:53:30.737336 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:30.737358 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:30.737774 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:30.738103 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:30.739793 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:30.741145 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:30.742890 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:30.743577 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:30.743709 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:30.748564 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:30.748572 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:30.748586 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:30.748622 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:30.751546 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:30.752312 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:30.752320 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:30.752333 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:30.752514 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:30.752911 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:30.752915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:30.752918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:30.753490 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:30.753791 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:30.754279 (+ 488us) server_negotiation.cc:299] Negotiation successful
0928 14:53:30.754712 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":360,"threads_started":1}
I20260928 14:53:30.757773 29239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:30.758140 29239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:30.758235 29239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6344471ac5fd4c6bb2e0e300525ba823, State: Initialized, Role: FOLLOWER
I20260928 14:53:30.758821 29239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:30.759027 29239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:53:30.759117 29239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:53:30.759239 29239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:30.761348 29239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:30.761796 29239 leader_election.cc:302] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6344471ac5fd4c6bb2e0e300525ba823; no voters:
I20260928 14:53:30.762306 29239 leader_election.cc:288] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:53:30.762434 29244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:53:30.763067 29244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 1 LEADER]: Becoming Leader. State: Replica: 6344471ac5fd4c6bb2e0e300525ba823, State: Running, Role: LEADER
I20260928 14:53:30.763614 29244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:30.764199 29239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:53:30.766498 29245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6344471ac5fd4c6bb2e0e300525ba823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } } }
I20260928 14:53:30.766521 29246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6344471ac5fd4c6bb2e0e300525ba823. Latest consensus state: current_term: 1 leader_uuid: "6344471ac5fd4c6bb2e0e300525ba823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } } }
I20260928 14:53:30.766937 29245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:30.766985 29246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:30.768909 29252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:53:30.774199 29252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:53:30.782755 29252 catalog_manager.cc:1347] Generated new cluster ID: a8a85e0da3a34202a555421e0e577685
I20260928 14:53:30.782876 29252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:53:30.803419 29252 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:53:30.805460 29252 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:53:30.816210 29252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Generated new TSK 0
I20260928 14:53:30.817440 29252 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:53:30.879647 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38925
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:31.211548 29265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:31.211853 29265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:31.211977 29265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:31.219484 29265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:31.219605 29265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:31.219749 29265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:53:31.230123 29265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:31.231467 29265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:31.232833 29265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:31.245945 29271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:31.246004 29270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:31.246022 29273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:31.246809 29265 server_base.cc:1034] running on GCE node
I20260928 14:53:31.247418 29265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:31.248457 29265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:31.249665 29265 hybrid_clock.cc:648] HybridClock initialized: now 1790607211249613 us; error 47 us; skew 500 ppm
I20260928 14:53:31.249992 29265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:31 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607211, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:31.253119 29265 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:53:31.254525 29265 webserver.cc:460] Webserver started at http://127.14.241.193:40581/ using document root <none> and password file <none>
I20260928 14:53:31.255196 29265 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:31.255314 29265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:31.255591 29265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:31.258610 29265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "8f503ec409e142f9a31c44b82c3d538c"
format_stamp: "Formatted at 2026-09-28 14:53:31 on dist-test-slave-vk40"
server_key: "a087d588a2ba12a478eefd5f821ca959"
server_key_iv: "defaa60bf0dd1872b3c12862ddfff986"
server_key_version: "encryptionkey@0"
I20260928 14:53:31.259385 29265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "8f503ec409e142f9a31c44b82c3d538c"
format_stamp: "Formatted at 2026-09-28 14:53:31 on dist-test-slave-vk40"
server_key: "a087d588a2ba12a478eefd5f821ca959"
server_key_iv: "defaa60bf0dd1872b3c12862ddfff986"
server_key_version: "encryptionkey@0"
I20260928 14:53:31.264550 29265 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260928 14:53:31.268005 29280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:31.269456 29265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:53:31.269586 29265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "8f503ec409e142f9a31c44b82c3d538c"
format_stamp: "Formatted at 2026-09-28 14:53:31 on dist-test-slave-vk40"
server_key: "a087d588a2ba12a478eefd5f821ca959"
server_key_iv: "defaa60bf0dd1872b3c12862ddfff986"
server_key_version: "encryptionkey@0"
I20260928 14:53:31.269752 29265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:31.294612 29265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:31.298627 29265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:31.298874 29265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:31.299853 29265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:31.301342 29265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:31.301420 29265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:31.301548 29265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:31.301597 29265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:31.332556 29265 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:45341
I20260928 14:53:31.332609 29392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:45341 every 8 connection(s)
I20260928 14:53:31.333882 29265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:53:31.342523 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 29265
I20260928 14:53:31.342715 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:53:31.343327 15303 external_mini_cluster.cc:1428] Setting key 8aadffa28890388e52c4d775a8368373
Sep 28 14:53:31 dist-test-slave-vk40 krb5kdc[29159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607211, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:31.353147 29395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:31.336344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:55075 (local address 127.14.241.254:38925)
0928 14:53:31.336762 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:31.336767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:31.337741 (+ 974us) server_negotiation.cc:407] Connection header received
0928 14:53:31.339534 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:31.339540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:31.339618 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:31.339714 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:31.341301 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:31.341999 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:31.343626 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:31.344024 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:31.344133 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:31.347500 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:31.347505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:31.347508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:31.347539 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:31.349986 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:31.350696 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:31.350700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:31.350702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:31.350825 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:31.351237 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:31.351241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:31.351243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:31.351507 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:31.351638 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:31.352382 (+ 744us) server_negotiation.cc:299] Negotiation successful
0928 14:53:31.352610 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":205,"threads_started":1}
I20260928 14:53:31.354084 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:31.336901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:55075)
0928 14:53:31.337585 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:31.337630 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:31.339255 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:31.339977 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:31.339987 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:31.340562 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:31.341128 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:31.341145 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:31.342163 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:31.342167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:31.343485 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:31.343497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:31.344240 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:31.344246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:31.344501 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:31.345156 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:31.345179 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:31.347285 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:31.350123 (+ 2838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:31.350132 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:31.350149 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:31.350572 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:31.350928 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:31.350932 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:31.350936 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:31.351119 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:31.351640 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:31.351647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:31.351972 (+ 325us) client_negotiation.cc:769] Sending connection context
0928 14:53:31.352210 (+ 238us) client_negotiation.cc:240] Negotiation successful
0928 14:53:31.352772 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":246,"threads_started":1}
I20260928 14:53:31.355767 29393 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38925
I20260928 14:53:31.356092 29393 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:31.356887 29393 heartbeater.cc:511] Master 127.14.241.254:38925 requested a full tablet report, sending...
I20260928 14:53:31.359021 29204 ts_manager.cc:194] Registered new tserver with Master: 8f503ec409e142f9a31c44b82c3d538c (127.14.241.193:45341)
I20260928 14:53:31.360571 29204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:55075
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:53:31.402968 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38925
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:31.722630 29399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:31.722909 29399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:31.723002 29399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:31.730437 29399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:31.730546 29399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:31.730696 29399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:53:31.740429 29399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:31.741674 29399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:31.742966 29399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:31.755810 29404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:31.755826 29405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:31.755934 29407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:31.756647 29399 server_base.cc:1034] running on GCE node
I20260928 14:53:31.757215 29399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:31.758126 29399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:31.759334 29399 hybrid_clock.cc:648] HybridClock initialized: now 1790607211759289 us; error 34 us; skew 500 ppm
I20260928 14:53:31.759631 29399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:31 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607211, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:31.762604 29399 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:53:31.764094 29399 webserver.cc:460] Webserver started at http://127.14.241.194:39137/ using document root <none> and password file <none>
I20260928 14:53:31.764779 29399 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:31.764895 29399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:31.765195 29399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:31.768231 29399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "118b5b082c6f484493a5cc2ce461a504"
format_stamp: "Formatted at 2026-09-28 14:53:31 on dist-test-slave-vk40"
server_key: "155a2fc84a437fdd9baec48e56ef2aae"
server_key_iv: "b92431cb34676643049876ecd8ea4701"
server_key_version: "encryptionkey@0"
I20260928 14:53:31.769091 29399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "118b5b082c6f484493a5cc2ce461a504"
format_stamp: "Formatted at 2026-09-28 14:53:31 on dist-test-slave-vk40"
server_key: "155a2fc84a437fdd9baec48e56ef2aae"
server_key_iv: "b92431cb34676643049876ecd8ea4701"
server_key_version: "encryptionkey@0"
I20260928 14:53:31.774225 29399 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:53:31.778062 29414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:31.779402 29399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:53:31.779546 29399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "118b5b082c6f484493a5cc2ce461a504"
format_stamp: "Formatted at 2026-09-28 14:53:31 on dist-test-slave-vk40"
server_key: "155a2fc84a437fdd9baec48e56ef2aae"
server_key_iv: "b92431cb34676643049876ecd8ea4701"
server_key_version: "encryptionkey@0"
I20260928 14:53:31.779738 29399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:31.812877 29399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:31.817358 29399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:31.817646 29399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:31.818635 29399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:31.820067 29399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:31.820142 29399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:31.820254 29399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:31.820333 29399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260928 14:53:31.851030 29399 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:39749
I20260928 14:53:31.851118 29526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:39749 every 8 connection(s)
I20260928 14:53:31.852401 29399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:53:31.856725 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 29399
I20260928 14:53:31.856915 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:53:31.857447 15303 external_mini_cluster.cc:1428] Setting key 3f7005e2606955f7b184eea47cc50084
Sep 28 14:53:31 dist-test-slave-vk40 krb5kdc[29159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607211, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:31.874038 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:31.854929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:38939 (local address 127.14.241.254:38925)
0928 14:53:31.855368 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:31.855374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:31.856445 (+ 1071us) server_negotiation.cc:407] Connection header received
0928 14:53:31.858757 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:31.858761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:31.858830 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:31.858920 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:31.860907 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:31.861639 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:31.863702 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:31.864152 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:31.864250 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:31.868155 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:31.868162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:31.868167 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:31.868211 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:31.870733 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:31.871489 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:31.871493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:31.871495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:31.871626 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:31.872075 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:31.872079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:31.872081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:31.872396 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:31.872543 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:31.873328 (+ 785us) server_negotiation.cc:299] Negotiation successful
0928 14:53:31.873577 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":210,"threads_started":1}
I20260928 14:53:31.874994 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:31.855564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:38939)
0928 14:53:31.856243 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:31.856325 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:31.858454 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:31.859216 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:31.859228 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:31.859885 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:31.860725 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:31.860752 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:31.861794 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:31.861799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:31.863567 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:31.863584 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:31.864398 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:31.864405 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:31.864672 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:31.865404 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:31.865438 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:31.867912 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:31.870898 (+ 2986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:31.870908 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:31.870922 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:31.871354 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:31.871737 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:31.871742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:31.871746 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:31.871954 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:31.872520 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:31.872529 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:31.872925 (+ 396us) client_negotiation.cc:769] Sending connection context
0928 14:53:31.873179 (+ 254us) client_negotiation.cc:240] Negotiation successful
0928 14:53:31.873741 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":271,"threads_started":1}
I20260928 14:53:31.876436 29527 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38925
I20260928 14:53:31.876725 29527 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:31.877453 29527 heartbeater.cc:511] Master 127.14.241.254:38925 requested a full tablet report, sending...
I20260928 14:53:31.878969 29204 ts_manager.cc:194] Registered new tserver with Master: 118b5b082c6f484493a5cc2ce461a504 (127.14.241.194:39749)
I20260928 14:53:31.880139 29204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:38939
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:53:31.911522 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38925
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:32.232589 29533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:32.232892 29533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:32.233000 29533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:32.240345 29533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:32.240465 29533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:32.240583 29533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:53:32.250448 29533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:32.251713 29533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:32.253077 29533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:32.265883 29538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:32.265973 29541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:32.265925 29539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:32.266736 29533 server_base.cc:1034] running on GCE node
I20260928 14:53:32.267375 29533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:32.268366 29533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:32.269563 29533 hybrid_clock.cc:648] HybridClock initialized: now 1790607212269508 us; error 47 us; skew 500 ppm
I20260928 14:53:32.269872 29533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:32 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607212, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:32.272840 29533 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:53:32.274327 29533 webserver.cc:460] Webserver started at http://127.14.241.195:38693/ using document root <none> and password file <none>
I20260928 14:53:32.275095 29533 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:32.275243 29533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:32.275529 29533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:32.278434 29533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "f08758e4b30c4b4b91400a9d9f8408c0"
format_stamp: "Formatted at 2026-09-28 14:53:32 on dist-test-slave-vk40"
server_key: "23cd034ac9a03ecb931f48e0d0ac63c2"
server_key_iv: "ad9f3126e5bdbac3ef8cf7017e774ef6"
server_key_version: "encryptionkey@0"
I20260928 14:53:32.279177 29533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "f08758e4b30c4b4b91400a9d9f8408c0"
format_stamp: "Formatted at 2026-09-28 14:53:32 on dist-test-slave-vk40"
server_key: "23cd034ac9a03ecb931f48e0d0ac63c2"
server_key_iv: "ad9f3126e5bdbac3ef8cf7017e774ef6"
server_key_version: "encryptionkey@0"
I20260928 14:53:32.284325 29533 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260928 14:53:32.287796 29548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:32.289443 29533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260928 14:53:32.289618 29533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "f08758e4b30c4b4b91400a9d9f8408c0"
format_stamp: "Formatted at 2026-09-28 14:53:32 on dist-test-slave-vk40"
server_key: "23cd034ac9a03ecb931f48e0d0ac63c2"
server_key_iv: "ad9f3126e5bdbac3ef8cf7017e774ef6"
server_key_version: "encryptionkey@0"
I20260928 14:53:32.289852 29533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:32.302480 29533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:32.306476 29533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:32.306798 29533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:32.307760 29533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:32.309219 29533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:32.309301 29533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:32.309427 29533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:32.309480 29533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:32.340058 29533 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:38041
I20260928 14:53:32.340095 29660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:38041 every 8 connection(s)
I20260928 14:53:32.341430 29533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:53:32.343382 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 29533
I20260928 14:53:32.343564 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:53:32.344019 15303 external_mini_cluster.cc:1428] Setting key 09e72960e38a14e1b93562cafa8649e8
Sep 28 14:53:32 dist-test-slave-vk40 krb5kdc[29159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607212, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:32.362106 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.344172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:46277 (local address 127.14.241.254:38925)
0928 14:53:32.344413 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:32.344420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:32.345665 (+ 1245us) server_negotiation.cc:407] Connection header received
0928 14:53:32.347763 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:32.347768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:32.347842 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:32.347936 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:32.349623 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.350398 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.351964 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.352415 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.352515 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.356124 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:32.356129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:32.356132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:32.356169 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:32.358997 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:32.359843 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:32.359847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:32.359850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:32.359978 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:32.360476 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:32.360480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:32.360482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:32.360777 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:32.360935 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:32.361653 (+ 718us) server_negotiation.cc:299] Negotiation successful
0928 14:53:32.361872 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 14:53:32.363104 29663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.344721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:46277)
0928 14:53:32.345502 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:32.345546 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:32.347458 (+ 1912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:32.348195 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:32.348205 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:32.348769 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:32.349436 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.349456 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.350551 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.350554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.351822 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.351835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.352590 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.352595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.352795 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.353464 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:32.353488 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:32.355893 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:32.359146 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:32.359153 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:32.359168 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:32.359686 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:32.360103 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:32.360107 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:32.360109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:32.360374 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:32.360930 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:32.360936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:32.361294 (+ 358us) client_negotiation.cc:769] Sending connection context
0928 14:53:32.361534 (+ 240us) client_negotiation.cc:240] Negotiation successful
0928 14:53:32.361985 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":288,"threads_started":1}
I20260928 14:53:32.363996 29393 heartbeater.cc:503] Master 127.14.241.254:38925 was elected leader, sending a full tablet report...
I20260928 14:53:32.364802 29661 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38925
I20260928 14:53:32.365155 29661 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:32.366043 29661 heartbeater.cc:511] Master 127.14.241.254:38925 requested a full tablet report, sending...
I20260928 14:53:32.367456 29204 ts_manager.cc:194] Registered new tserver with Master: f08758e4b30c4b4b91400a9d9f8408c0 (127.14.241.195:38041)
I20260928 14:53:32.368314 29204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:46277
I20260928 14:53:32.371874 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:53:32.389058 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.376415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52010 (local address 127.14.241.254:38925)
0928 14:53:32.376571 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:32.376578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:32.376998 (+ 420us) server_negotiation.cc:407] Connection header received
0928 14:53:32.377159 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:32.377163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:32.377235 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:32.377325 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:32.378666 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.379397 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.380960 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.381373 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.381485 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.384064 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:32.384070 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:32.384073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:32.384108 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:32.386702 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:32.387343 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:32.387347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:32.387350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:32.387492 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:32.387777 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:32.387780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:32.387782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:32.388091 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:32.388331 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:32.388562 (+ 231us) server_negotiation.cc:299] Negotiation successful
0928 14:53:32.388730 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 14:53:32.392972 29204 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:53:32.399971 29204 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:53:32.423797 29673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.412579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50332 (local address 127.14.241.193:45341)
0928 14:53:32.414834 (+ 2255us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:32.414842 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:32.414872 (+ 30us) server_negotiation.cc:407] Connection header received
0928 14:53:32.414973 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:32.414982 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:32.415278 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:32.416314 (+ 1036us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:32.417005 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.418597 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.421444 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.422621 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.422818 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.423018 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:32.423326 (+ 308us) server_negotiation.cc:299] Negotiation successful
0928 14:53:32.423488 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":282,"threads_started":1}
I20260928 14:53:32.423875 29674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.412626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:45341 (local address 127.0.0.1:50332)
0928 14:53:32.413868 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:32.413922 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:32.414122 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:32.415835 (+ 1713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:32.415843 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:32.415873 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:32.416780 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.416800 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.418833 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.418843 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.421237 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.421250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.422856 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.422862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.423067 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.423122 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 14:53:32.423227 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 14:53:32.423433 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"mutex_wait_us":33,"thread_start_us":808,"threads_started":1}
I20260928 14:53:32.424660 29676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.413798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52902 (local address 127.14.241.194:39749)
0928 14:53:32.414677 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:32.414686 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:32.415235 (+ 549us) server_negotiation.cc:407] Connection header received
0928 14:53:32.415337 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:32.415344 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:32.415574 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:32.415790 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:32.418232 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.419480 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.421765 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.423247 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.423380 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.423545 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:32.424152 (+ 607us) server_negotiation.cc:299] Negotiation successful
0928 14:53:32.424359 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":623,"threads_started":1}
I20260928 14:53:32.425262 29677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.413529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:39749 (local address 127.0.0.1:52902)
0928 14:53:32.414991 (+ 1462us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:32.415007 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:32.415121 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:32.416055 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:32.416059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:32.416079 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:32.416965 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.416978 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.419614 (+ 2636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.419619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.421553 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.421569 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.423757 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.423764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.423947 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.423978 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 14:53:32.424783 (+ 805us) client_negotiation.cc:240] Negotiation successful
0928 14:53:32.424928 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1349,"thread_start_us":3585,"threads_started":1}
I20260928 14:53:32.425439 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.414900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.14.241.195:38041)
0928 14:53:32.416626 (+ 1726us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:32.416635 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:32.417645 (+ 1010us) server_negotiation.cc:407] Connection header received
0928 14:53:32.417756 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:32.417764 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:32.418001 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:32.418178 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:32.418750 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.420388 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.422918 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.424363 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.424557 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.424724 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:32.424980 (+ 256us) server_negotiation.cc:299] Negotiation successful
0928 14:53:32.425149 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1554,"thread_start_us":1399,"threads_started":1}
I20260928 14:53:32.425949 29675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.414346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:38041 (local address 127.0.0.1:41072)
0928 14:53:32.417390 (+ 3044us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:32.417410 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:32.417518 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:32.418175 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:32.418179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:32.418203 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:32.418550 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.418560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.420534 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.420540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.422697 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.422714 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.424559 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.424565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.424741 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.424772 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 14:53:32.424854 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:53:32.424976 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2927,"thread_start_us":707,"threads_started":1}
I20260928 14:53:32.428306 29328 tablet_service.cc:1467] Processing CreateTablet for tablet 3a90c6d604da4851a6de8ef43636f1bd (DEFAULT_TABLE table=test-table [id=4ff2cd2c7352435fab40fb9a99ec4d4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:32.429580 29328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a90c6d604da4851a6de8ef43636f1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:32.429164 29462 tablet_service.cc:1467] Processing CreateTablet for tablet 3a90c6d604da4851a6de8ef43636f1bd (DEFAULT_TABLE table=test-table [id=4ff2cd2c7352435fab40fb9a99ec4d4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:32.429550 29596 tablet_service.cc:1467] Processing CreateTablet for tablet 3a90c6d604da4851a6de8ef43636f1bd (DEFAULT_TABLE table=test-table [id=4ff2cd2c7352435fab40fb9a99ec4d4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:32.430548 29462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a90c6d604da4851a6de8ef43636f1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:32.430836 29596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a90c6d604da4851a6de8ef43636f1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:32.441280 29679 tablet_bootstrap.cc:492] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c: Bootstrap starting.
I20260928 14:53:32.445508 29679 tablet_bootstrap.cc:654] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:32.446652 29679 log.cc:824] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:32.447659 29681 tablet_bootstrap.cc:492] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0: Bootstrap starting.
I20260928 14:53:32.449975 29680 tablet_bootstrap.cc:492] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504: Bootstrap starting.
I20260928 14:53:32.451776 29679 tablet_bootstrap.cc:492] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c: No bootstrap required, opened a new log
I20260928 14:53:32.451879 29681 tablet_bootstrap.cc:654] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:32.452059 29679 ts_tablet_manager.cc:1397] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260928 14:53:32.453262 29681 log.cc:824] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:32.454100 29680 tablet_bootstrap.cc:654] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:32.455497 29680 log.cc:824] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:32.458630 29680 tablet_bootstrap.cc:492] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504: No bootstrap required, opened a new log
I20260928 14:53:32.458899 29680 ts_tablet_manager.cc:1397] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 14:53:32.458918 29681 tablet_bootstrap.cc:492] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0: No bootstrap required, opened a new log
I20260928 14:53:32.459174 29681 ts_tablet_manager.cc:1397] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260928 14:53:32.461869 29679 raft_consensus.cc:348] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } }
I20260928 14:53:32.462419 29679 raft_consensus.cc:374] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:32.462512 29679 raft_consensus.cc:729] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f503ec409e142f9a31c44b82c3d538c, State: Initialized, Role: FOLLOWER
I20260928 14:53:32.463173 29679 consensus_queue.cc:260] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } }
I20260928 14:53:32.465209 29679 ts_tablet_manager.cc:1428] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260928 14:53:32.468228 29680 raft_consensus.cc:348] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } }
I20260928 14:53:32.468779 29680 raft_consensus.cc:374] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:32.468894 29680 raft_consensus.cc:729] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118b5b082c6f484493a5cc2ce461a504, State: Initialized, Role: FOLLOWER
I20260928 14:53:32.469497 29680 consensus_queue.cc:260] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } }
I20260928 14:53:32.469767 29681 raft_consensus.cc:348] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } }
I20260928 14:53:32.470222 29681 raft_consensus.cc:374] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:32.470335 29681 raft_consensus.cc:729] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08758e4b30c4b4b91400a9d9f8408c0, State: Initialized, Role: FOLLOWER
I20260928 14:53:32.470920 29527 heartbeater.cc:503] Master 127.14.241.254:38925 was elected leader, sending a full tablet report...
I20260928 14:53:32.470939 29681 consensus_queue.cc:260] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } }
I20260928 14:53:32.471974 29661 heartbeater.cc:503] Master 127.14.241.254:38925 was elected leader, sending a full tablet report...
I20260928 14:53:32.472784 29681 ts_tablet_manager.cc:1428] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0: Time spent starting tablet: real 0.013s user 0.007s sys 0.007s
I20260928 14:53:32.472996 29680 ts_tablet_manager.cc:1428] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260928 14:53:32.484009 29685 raft_consensus.cc:482] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:53:32.484319 29685 raft_consensus.cc:504] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } }
I20260928 14:53:32.486027 29685 leader_election.cc:288] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f08758e4b30c4b4b91400a9d9f8408c0 (127.14.241.195:38041), 118b5b082c6f484493a5cc2ce461a504 (127.14.241.194:39749)
I20260928 14:53:32.492671 29676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.486417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:48863 (local address 127.14.241.194:39749)
0928 14:53:32.486628 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:32.486635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:32.487440 (+ 805us) server_negotiation.cc:407] Connection header received
0928 14:53:32.487600 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:32.487604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:32.487665 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:32.487749 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:32.488408 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.489121 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.490922 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.491709 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.491844 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.491919 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:32.492273 (+ 354us) server_negotiation.cc:299] Negotiation successful
0928 14:53:32.492430 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 14:53:32.492672 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.486407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:54197 (local address 127.14.241.195:38041)
0928 14:53:32.486672 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:32.486693 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:32.487329 (+ 636us) server_negotiation.cc:407] Connection header received
0928 14:53:32.487545 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:32.487550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:32.487622 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:32.487686 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:32.488372 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.489142 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.490963 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.491713 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.491866 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.491940 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:32.492307 (+ 367us) server_negotiation.cc:299] Negotiation successful
0928 14:53:32.492433 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260928 14:53:32.492838 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.486428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:38041 (local address 127.14.241.193:54197)
0928 14:53:32.487223 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:32.487242 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:32.487385 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:32.487732 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:32.487736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:32.487784 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:32.488224 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.488234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.489266 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.489270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.490813 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.490825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.491977 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.491982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.492137 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.492160 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 14:53:32.492241 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 14:53:32.492390 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":105,"thread_start_us":474,"threads_started":1}
I20260928 14:53:32.493167 29689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.486469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:39749 (local address 127.14.241.193:48863)
0928 14:53:32.487347 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:32.487362 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:32.487477 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:32.487774 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:32.487778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:32.487803 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:32.488239 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.488248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.489251 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.489256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.490762 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:32.490777 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.491930 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.491938 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:32.492110 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.492144 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 14:53:32.492218 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 14:53:32.492387 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"mutex_wait_us":22,"thread_start_us":456,"threads_started":1}
I20260928 14:53:32.493861 29616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a90c6d604da4851a6de8ef43636f1bd" candidate_uuid: "8f503ec409e142f9a31c44b82c3d538c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" is_pre_election: true
I20260928 14:53:32.494208 29482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a90c6d604da4851a6de8ef43636f1bd" candidate_uuid: "8f503ec409e142f9a31c44b82c3d538c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118b5b082c6f484493a5cc2ce461a504" is_pre_election: true
I20260928 14:53:32.494459 29616 raft_consensus.cc:2454] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f503ec409e142f9a31c44b82c3d538c in term 0.
I20260928 14:53:32.494692 29482 raft_consensus.cc:2454] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f503ec409e142f9a31c44b82c3d538c in term 0.
I20260928 14:53:32.495144 29284 leader_election.cc:302] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f503ec409e142f9a31c44b82c3d538c, f08758e4b30c4b4b91400a9d9f8408c0; no voters:
I20260928 14:53:32.495527 29685 raft_consensus.cc:2781] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:53:32.495648 29685 raft_consensus.cc:482] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:53:32.495726 29685 raft_consensus.cc:3037] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:32.497927 29685 raft_consensus.cc:504] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } }
I20260928 14:53:32.498476 29685 leader_election.cc:288] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [CANDIDATE]: Term 1 election: Requested vote from peers f08758e4b30c4b4b91400a9d9f8408c0 (127.14.241.195:38041), 118b5b082c6f484493a5cc2ce461a504 (127.14.241.194:39749)
I20260928 14:53:32.498965 29616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a90c6d604da4851a6de8ef43636f1bd" candidate_uuid: "8f503ec409e142f9a31c44b82c3d538c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08758e4b30c4b4b91400a9d9f8408c0"
I20260928 14:53:32.499054 29482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a90c6d604da4851a6de8ef43636f1bd" candidate_uuid: "8f503ec409e142f9a31c44b82c3d538c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118b5b082c6f484493a5cc2ce461a504"
I20260928 14:53:32.499150 29616 raft_consensus.cc:3037] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:32.499284 29482 raft_consensus.cc:3037] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:32.501539 29482 raft_consensus.cc:2454] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f503ec409e142f9a31c44b82c3d538c in term 1.
I20260928 14:53:32.501554 29616 raft_consensus.cc:2454] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f503ec409e142f9a31c44b82c3d538c in term 1.
I20260928 14:53:32.501986 29284 leader_election.cc:302] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 118b5b082c6f484493a5cc2ce461a504, 8f503ec409e142f9a31c44b82c3d538c, f08758e4b30c4b4b91400a9d9f8408c0; no voters:
I20260928 14:53:32.502313 29685 raft_consensus.cc:2781] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:53:32.502727 29685 raft_consensus.cc:686] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 1 LEADER]: Becoming Leader. State: Replica: 8f503ec409e142f9a31c44b82c3d538c, State: Running, Role: LEADER
I20260928 14:53:32.503214 29685 consensus_queue.cc:237] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } }
I20260928 14:53:32.508141 29203 catalog_manager.cc:5495] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f503ec409e142f9a31c44b82c3d538c (127.14.241.193). New cstate: current_term: 1 leader_uuid: "8f503ec409e142f9a31c44b82c3d538c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } health_report { overall_health: UNKNOWN } } }
I20260928 14:53:32.525856 29673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.520246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50344 (local address 127.14.241.193:45341)
0928 14:53:32.520523 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:32.520528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:32.520548 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:53:32.520750 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:32.520755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:32.520856 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:32.520995 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:53:32.521392 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.522095 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.523473 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.523906 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.524020 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.524730 (+ 710us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:53:32.525259 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:53:32.525403 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:32.525488 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 14:53:32.525574 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260928 14:53:32.534394 29616 raft_consensus.cc:1261] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0 [term 1 FOLLOWER]: Refusing update from remote peer 8f503ec409e142f9a31c44b82c3d538c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:53:32.534392 29482 raft_consensus.cc:1261] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504 [term 1 FOLLOWER]: Refusing update from remote peer 8f503ec409e142f9a31c44b82c3d538c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:53:32.535380 29690 consensus_queue.cc:1035] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [LEADER]: Connected to new peer: Peer: permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:53:32.535737 29685 consensus_queue.cc:1035] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:53:32.547613 29693 mvcc.cc:205] Tried to move back new op lower bound from 7334327142531125248 to 7334327142416650240. Current Snapshot: MvccSnapshot[applied={T|T < 7334327142531125248}]
I20260928 14:53:32.562006 29204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:53:32.562335 29204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:53:32.566241 29204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Sending DeleteTablet for 3 replicas of tablet 3a90c6d604da4851a6de8ef43636f1bd
I20260928 14:53:32.567531 29328 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a90c6d604da4851a6de8ef43636f1bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:32 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:50332
I20260928 14:53:32.567602 29596 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a90c6d604da4851a6de8ef43636f1bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:32 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:41072
I20260928 14:53:32.567860 29462 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a90c6d604da4851a6de8ef43636f1bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:32 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:52902
I20260928 14:53:32.568009 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 29265
I20260928 14:53:32.577646 29705 tablet_replica.cc:330] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504: stopping tablet replica
I20260928 14:53:32.578370 29705 raft_consensus.cc:2229] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:53:32.579077 29705 raft_consensus.cc:2258] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:53:32.581429 29705 ts_tablet_manager.cc:1905] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:53:32.582732 29704 tablet_replica.cc:330] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c: stopping tablet replica
I20260928 14:53:32.583187 29704 raft_consensus.cc:2229] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:53:32.587471 29704 raft_consensus.cc:2258] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:53:32.590054 29703 tablet_replica.cc:330] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0: stopping tablet replica
I20260928 14:53:32.590495 29705 ts_tablet_manager.cc:1918] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:53:32.590525 29703 raft_consensus.cc:2229] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:53:32.590756 29705 log.cc:1196] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/3a90c6d604da4851a6de8ef43636f1bd
I20260928 14:53:32.591059 29703 raft_consensus.cc:2258] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:53:32.591406 29704 ts_tablet_manager.cc:1905] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:53:32.591449 29705 ts_tablet_manager.cc:1939] T 3a90c6d604da4851a6de8ef43636f1bd P 118b5b082c6f484493a5cc2ce461a504: Deleting consensus metadata
I20260928 14:53:32.593449 29188 catalog_manager.cc:4841] TS 118b5b082c6f484493a5cc2ce461a504 (127.14.241.194:39749): tablet 3a90c6d604da4851a6de8ef43636f1bd (table test-table [id=4ff2cd2c7352435fab40fb9a99ec4d4e]) successfully deleted
Sep 28 14:53:32 dist-test-slave-vk40 krb5kdc[29159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:53:32.595902 29703 ts_tablet_manager.cc:1905] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:53:32.598312 29704 ts_tablet_manager.cc:1918] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:53:32.598479 29704 log.cc:1196] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/3a90c6d604da4851a6de8ef43636f1bd
I20260928 14:53:32.599076 29704 ts_tablet_manager.cc:1939] T 3a90c6d604da4851a6de8ef43636f1bd P 8f503ec409e142f9a31c44b82c3d538c: Deleting consensus metadata
I20260928 14:53:32.600980 29191 catalog_manager.cc:4841] TS 8f503ec409e142f9a31c44b82c3d538c (127.14.241.193:45341): tablet 3a90c6d604da4851a6de8ef43636f1bd (table test-table [id=4ff2cd2c7352435fab40fb9a99ec4d4e]) successfully deleted
I20260928 14:53:32.602871 29703 ts_tablet_manager.cc:1918] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:53:32.603034 29703 log.cc:1196] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/3a90c6d604da4851a6de8ef43636f1bd
I20260928 14:53:32.603562 29703 ts_tablet_manager.cc:1939] T 3a90c6d604da4851a6de8ef43636f1bd P f08758e4b30c4b4b91400a9d9f8408c0: Deleting consensus metadata
I20260928 14:53:32.605597 29191 catalog_manager.cc:4841] TS f08758e4b30c4b4b91400a9d9f8408c0 (127.14.241.195:38041): tablet 3a90c6d604da4851a6de8ef43636f1bd (table test-table [id=4ff2cd2c7352435fab40fb9a99ec4d4e]) successfully deleted
I20260928 14:53:32.606519 29673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.569219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50348 (local address 127.14.241.193:45341)
0928 14:53:32.569403 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:32.569410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:32.579479 (+ 10069us) server_negotiation.cc:407] Connection header received
0928 14:53:32.580032 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:32.580039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:32.580127 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:32.580205 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:32.583754 (+ 3549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.585002 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.587889 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.588645 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.588761 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.599903 (+ 11142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:32.599917 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:32.599935 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:32.599984 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:32.603385 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:32.604189 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:32.604204 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:32.604206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:32.604404 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:32.604816 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:32.604820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:32.604822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:32.605182 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:32.605514 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:32.605929 (+ 415us) server_negotiation.cc:299] Negotiation successful
0928 14:53:32.606180 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:53:32.607362 29388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:32.708896 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 29265
I20260928 14:53:32.725220 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 29399
Sep 28 14:53:32 dist-test-slave-vk40 krb5kdc[29159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:53:32.741088 29676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.726238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55192 (local address 127.14.241.194:39749)
0928 14:53:32.726444 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:32.726448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:32.726464 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:53:32.726519 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:32.726522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:32.726591 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:32.726703 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:32.727947 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.728766 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.730098 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.730524 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.730633 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.735381 (+ 4748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:32.735398 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:32.735413 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:32.735451 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:32.738184 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:32.738888 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:32.738896 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:32.738898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:32.739031 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:32.739394 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:32.739398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:32.739400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:32.739777 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:32.740055 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:32.740600 (+ 545us) server_negotiation.cc:299] Negotiation successful
0928 14:53:32.740823 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 14:53:32.741806 29522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:32.830272 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 29399
I20260928 14:53:32.844230 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 29533
Sep 28 14:53:32 dist-test-slave-vk40 krb5kdc[29159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:53:32.859316 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:32.845112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53054 (local address 127.14.241.195:38041)
0928 14:53:32.845341 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:32.845347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:32.845366 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:53:32.845427 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:32.845430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:32.845503 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:32.845585 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:32.846957 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.847694 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.849119 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:32.849535 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:32.849642 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:32.853896 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:32.853913 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:32.853929 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:32.853965 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:32.856643 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:32.857316 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:32.857324 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:32.857327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:32.857460 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:32.857821 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:32.857825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:32.857827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:32.858182 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:32.858468 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:32.858776 (+ 308us) server_negotiation.cc:299] Negotiation successful
0928 14:53:32.858991 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 14:53:32.859994 29656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:32.948534 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 29533
I20260928 14:53:32.962596 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 29172
I20260928 14:53:32.963792 29234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:33.077301 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 29172
I20260928 14:53:33.095103 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38925
--webserver_interface=127.14.241.254
--webserver_port=37827
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:38925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:33.423722 29711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:33.424099 29711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:33.424230 29711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:33.431775 29711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:53:33.431919 29711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:33.431999 29711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:33.432077 29711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:53:33.432138 29711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:53:33.442222 29711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:38925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=37827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:33.443559 29711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:33.444905 29711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:33.469019 29718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:33.469044 29717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:33.469101 29720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:33.469945 29711 server_base.cc:1034] running on GCE node
I20260928 14:53:33.470860 29711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:33.472539 29711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:33.473770 29711 hybrid_clock.cc:648] HybridClock initialized: now 1790607213473714 us; error 50 us; skew 500 ppm
I20260928 14:53:33.474208 29711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:33 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607213, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:33.477803 29711 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:53:33.479768 29711 webserver.cc:460] Webserver started at http://127.14.241.254:37827/ using document root <none> and password file <none>
I20260928 14:53:33.480597 29711 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:33.480746 29711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:33.486912 29711 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 14:53:33.490255 29727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:33.491878 29711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:53:33.492074 29711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "6344471ac5fd4c6bb2e0e300525ba823"
format_stamp: "Formatted at 2026-09-28 14:53:30 on dist-test-slave-vk40"
server_key: "528c0f03898e3b5322ca8303892e2bc3"
server_key_iv: "63611d91a577b0da8bee6db70f6e51c4"
server_key_version: "encryptionkey@0"
I20260928 14:53:33.493263 29711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:33.513165 29711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:33.517814 29711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:33.518090 29711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:33.536784 29711 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:38925
I20260928 14:53:33.536791 29778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:38925 every 8 connection(s)
I20260928 14:53:33.538369 29711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:53:33.545488 29779 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:53:33.547670 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 29711
I20260928 14:53:33.548517 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:45341
--local_ip_for_outbound_sockets=127.14.241.193
--tserver_master_addrs=127.14.241.254:38925
--webserver_port=40581
--webserver_interface=127.14.241.193
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:53:33.548982 29779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Bootstrap starting.
I20260928 14:53:33.569191 29779 log.cc:824] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:33.582123 29779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:53:33.582656 29779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Bootstrap complete.
I20260928 14:53:33.591857 29779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:33.592736 29779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6344471ac5fd4c6bb2e0e300525ba823, State: Initialized, Role: FOLLOWER
I20260928 14:53:33.593358 29779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:33.593572 29779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:53:33.593689 29779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:53:33.593842 29779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:53:33.597815 29779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:33.598346 29779 leader_election.cc:302] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6344471ac5fd4c6bb2e0e300525ba823; no voters:
I20260928 14:53:33.598941 29779 leader_election.cc:288] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:53:33.599080 29783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:53:33.601262 29779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:53:33.605758 29783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 2 LEADER]: Becoming Leader. State: Replica: 6344471ac5fd4c6bb2e0e300525ba823, State: Running, Role: LEADER
I20260928 14:53:33.606470 29783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:33.609433 29784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6344471ac5fd4c6bb2e0e300525ba823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } } }
I20260928 14:53:33.609879 29784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:33.610451 29785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6344471ac5fd4c6bb2e0e300525ba823. Latest consensus state: current_term: 2 leader_uuid: "6344471ac5fd4c6bb2e0e300525ba823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } } }
I20260928 14:53:33.610721 29785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:33.617697 29796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:53:33.627590 29796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:53:33.629132 29796 catalog_manager.cc:1259] Loaded cluster ID: a8a85e0da3a34202a555421e0e577685
I20260928 14:53:33.629242 29796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:53:33.632478 29796 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260928 14:53:33.651296 29796 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260928 14:53:33.660583 29796 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:53:33.662313 29796 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Loaded TSK: 0
I20260928 14:53:33.663841 29796 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:53:33.893036 29781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:33.893327 29781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:33.893462 29781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:33.901111 29781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:33.901239 29781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:33.901393 29781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:53:33.912025 29781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:45341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=40581
--tserver_master_addrs=127.14.241.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:33.913378 29781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:33.914731 29781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:33.929273 29807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:33.929806 29808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:33.929385 29810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:33.930433 29781 server_base.cc:1034] running on GCE node
I20260928 14:53:33.931074 29781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:33.932097 29781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:33.933316 29781 hybrid_clock.cc:648] HybridClock initialized: now 1790607213933261 us; error 58 us; skew 500 ppm
I20260928 14:53:33.933624 29781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:33 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607213, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:33.940627 29781 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:53:33.942220 29781 webserver.cc:460] Webserver started at http://127.14.241.193:40581/ using document root <none> and password file <none>
I20260928 14:53:33.942906 29781 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:33.943040 29781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:33.948591 29781 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260928 14:53:33.951610 29817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:33.953044 29781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:53:33.953202 29781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "8f503ec409e142f9a31c44b82c3d538c"
format_stamp: "Formatted at 2026-09-28 14:53:31 on dist-test-slave-vk40"
server_key: "a087d588a2ba12a478eefd5f821ca959"
server_key_iv: "defaa60bf0dd1872b3c12862ddfff986"
server_key_version: "encryptionkey@0"
I20260928 14:53:33.954172 29781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:33.995074 29781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:33.999451 29781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:33.999711 29781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:34.000775 29781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:34.002193 29781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:34.002303 29781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:34.002408 29781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:34.002481 29781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:34.033209 29781 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:45341
I20260928 14:53:34.033299 29929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:45341 every 8 connection(s)
I20260928 14:53:34.034740 29781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:53:34.041580 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 29781
I20260928 14:53:34.042357 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:39749
--local_ip_for_outbound_sockets=127.14.241.194
--tserver_master_addrs=127.14.241.254:38925
--webserver_port=39137
--webserver_interface=127.14.241.194
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:53:34 dist-test-slave-vk40 krb5kdc[29159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607213, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:34.059018 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:34.037633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:54845 (local address 127.14.241.254:38925)
0928 14:53:34.038161 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:34.038174 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:34.038936 (+ 762us) server_negotiation.cc:407] Connection header received
0928 14:53:34.041129 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:34.041150 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:34.041661 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:34.042094 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:34.043588 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.045346 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.046950 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.047878 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.048060 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:34.051591 (+ 3531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:34.051600 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:34.051615 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:34.051653 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:34.054408 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:34.055254 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:34.055260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:34.055266 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:34.055429 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:34.055846 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:34.055849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:34.055850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:34.056464 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:34.056651 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:34.057495 (+ 844us) server_negotiation.cc:299] Negotiation successful
0928 14:53:34.057893 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":225,"threads_started":1}
I20260928 14:53:34.059127 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:34.037886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:54845)
0928 14:53:34.038720 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:34.038776 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:34.040526 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:34.042196 (+ 1670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:34.042204 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:34.042789 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:34.043345 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:34.043363 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.045520 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.045523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:34.046805 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:34.046818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.048093 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.048097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:34.048325 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:34.048967 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:34.048991 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:34.051316 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:34.054562 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:34.054570 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:34.054586 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:34.055132 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:34.055531 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:34.055535 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:34.055538 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:34.055754 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:34.056612 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:34.056619 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:34.057003 (+ 384us) client_negotiation.cc:769] Sending connection context
0928 14:53:34.057312 (+ 309us) client_negotiation.cc:240] Negotiation successful
0928 14:53:34.057889 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":324,"threads_started":1}
I20260928 14:53:34.062352 29930 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38925
I20260928 14:53:34.062652 29930 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:34.063396 29930 heartbeater.cc:511] Master 127.14.241.254:38925 requested a full tablet report, sending...
I20260928 14:53:34.065919 29744 ts_manager.cc:194] Registered new tserver with Master: 8f503ec409e142f9a31c44b82c3d538c (127.14.241.193:45341)
I20260928 14:53:34.068120 29744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:54845
W20260928 14:53:34.371454 29934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:34.371773 29934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:34.371896 29934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:34.379858 29934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:34.379979 29934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:34.380133 29934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:53:34.390247 29934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:39749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=39137
--tserver_master_addrs=127.14.241.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:34.391559 29934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:34.392956 29934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:34.406589 29939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:34.406520 29940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:34.406487 29942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:34.406908 29934 server_base.cc:1034] running on GCE node
I20260928 14:53:34.407761 29934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:34.408810 29934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:34.410053 29934 hybrid_clock.cc:648] HybridClock initialized: now 1790607214409986 us; error 49 us; skew 500 ppm
I20260928 14:53:34.410389 29934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:34 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607214, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:34.413725 29934 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:53:34.415230 29934 webserver.cc:460] Webserver started at http://127.14.241.194:39137/ using document root <none> and password file <none>
I20260928 14:53:34.415947 29934 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:34.416078 29934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:34.421649 29934 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260928 14:53:34.424763 29949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:34.426301 29934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 14:53:34.426505 29934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "118b5b082c6f484493a5cc2ce461a504"
format_stamp: "Formatted at 2026-09-28 14:53:31 on dist-test-slave-vk40"
server_key: "155a2fc84a437fdd9baec48e56ef2aae"
server_key_iv: "b92431cb34676643049876ecd8ea4701"
server_key_version: "encryptionkey@0"
I20260928 14:53:34.427567 29934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:34.457439 29934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:34.463153 29934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:34.463405 29934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:34.464434 29934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:34.465860 29934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:34.465946 29934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:34.466053 29934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:34.466128 29934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:34.497404 29934 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:39749
I20260928 14:53:34.497612 30061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:39749 every 8 connection(s)
I20260928 14:53:34.498974 29934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:53:34.508781 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 29934
I20260928 14:53:34.509371 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:38041
--local_ip_for_outbound_sockets=127.14.241.195
--tserver_master_addrs=127.14.241.254:38925
--webserver_port=38693
--webserver_interface=127.14.241.195
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:53:34 dist-test-slave-vk40 krb5kdc[29159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607214, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:34.519255 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:34.501568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:60259 (local address 127.14.241.254:38925)
0928 14:53:34.501752 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:34.501758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:34.503109 (+ 1351us) server_negotiation.cc:407] Connection header received
0928 14:53:34.505035 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:34.505039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:34.505114 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:34.505210 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:34.506750 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.507564 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.509359 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.509852 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.509972 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:34.513486 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:34.513491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:34.513494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:34.513530 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:34.515999 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:34.516837 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:34.516841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:34.516844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:34.516973 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:34.517428 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:34.517432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:34.517434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:34.517738 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:34.517877 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:34.518650 (+ 773us) server_negotiation.cc:299] Negotiation successful
0928 14:53:34.518915 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 14:53:34.520267 30064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:34.502122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:60259)
0928 14:53:34.502912 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:34.502969 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:34.504772 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:34.505419 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:34.505428 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:34.505965 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:34.506589 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:34.506607 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.507741 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.507745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:34.509181 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:34.509214 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.510085 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.510092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:34.510330 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:34.510989 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:34.511013 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:34.513236 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:34.516159 (+ 2923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:34.516168 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:34.516187 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:34.516691 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:34.517089 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:34.517094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:34.517098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:34.517304 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:34.517872 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:34.517881 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:34.518257 (+ 376us) client_negotiation.cc:769] Sending connection context
0928 14:53:34.518511 (+ 254us) client_negotiation.cc:240] Negotiation successful
0928 14:53:34.519078 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":315,"threads_started":1}
I20260928 14:53:34.521863 30062 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38925
I20260928 14:53:34.522239 30062 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:34.523190 30062 heartbeater.cc:511] Master 127.14.241.254:38925 requested a full tablet report, sending...
I20260928 14:53:34.524981 29744 ts_manager.cc:194] Registered new tserver with Master: 118b5b082c6f484493a5cc2ce461a504 (127.14.241.194:39749)
I20260928 14:53:34.526140 29744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:60259
W20260928 14:53:34.836086 30065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:34.836382 30065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:34.836485 30065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:34.843611 30065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:34.843734 30065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:34.843848 30065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:53:34.853595 30065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:38041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=38693
--tserver_master_addrs=127.14.241.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:34.854830 30065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:34.856050 30065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:34.868876 30073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:34.868902 30071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:34.869124 30070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:34.869616 30065 server_base.cc:1034] running on GCE node
I20260928 14:53:34.870167 30065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:34.871076 30065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:34.872350 30065 hybrid_clock.cc:648] HybridClock initialized: now 1790607214872246 us; error 56 us; skew 500 ppm
I20260928 14:53:34.872743 30065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:34 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607214, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:34.875998 30065 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:53:34.877470 30065 webserver.cc:460] Webserver started at http://127.14.241.195:38693/ using document root <none> and password file <none>
I20260928 14:53:34.878145 30065 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:34.878258 30065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:34.883571 30065 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 14:53:34.886282 30080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:34.887606 30065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:53:34.887764 30065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "f08758e4b30c4b4b91400a9d9f8408c0"
format_stamp: "Formatted at 2026-09-28 14:53:32 on dist-test-slave-vk40"
server_key: "23cd034ac9a03ecb931f48e0d0ac63c2"
server_key_iv: "ad9f3126e5bdbac3ef8cf7017e774ef6"
server_key_version: "encryptionkey@0"
I20260928 14:53:34.888788 30065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:34.916137 30065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:34.920176 30065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:34.920461 30065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:34.921456 30065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:34.922820 30065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:34.922909 30065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:34.922986 30065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:34.923039 30065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:34.953521 30065 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:38041
I20260928 14:53:34.953593 30192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:38041 every 8 connection(s)
I20260928 14:53:34.954917 30065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:53:34.962607 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 30065
Sep 28 14:53:34 dist-test-slave-vk40 krb5kdc[29159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607214, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:34.976882 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:34.957473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:37561 (local address 127.14.241.254:38925)
0928 14:53:34.957676 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:34.957682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:34.958876 (+ 1194us) server_negotiation.cc:407] Connection header received
0928 14:53:34.960776 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:34.960781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:34.960854 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:34.960950 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:34.962792 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.963613 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.965819 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.966262 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.966369 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:34.970542 (+ 4173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:34.970548 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:34.970553 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:34.970592 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:34.973510 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:34.974330 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:34.974335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:34.974339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:34.974506 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:34.974947 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:34.974952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:34.974955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:34.975334 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:34.975483 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:34.976274 (+ 791us) server_negotiation.cc:299] Negotiation successful
0928 14:53:34.976548 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 14:53:34.978070 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:34.957960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:37561)
0928 14:53:34.958700 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:34.958753 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:34.960445 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:34.961226 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:34.961234 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:34.961784 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:34.962536 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:34.962565 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.963775 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.963781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:34.965652 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:34.965670 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.966517 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.966524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:34.966800 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:34.967446 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:34.967473 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:34.970301 (+ 2828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:34.973642 (+ 3341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:34.973650 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:34.973664 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:34.974197 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:34.974616 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:34.974619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:34.974621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:34.974806 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:34.975466 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:34.975472 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:34.975865 (+ 393us) client_negotiation.cc:769] Sending connection context
0928 14:53:34.976183 (+ 318us) client_negotiation.cc:240] Negotiation successful
0928 14:53:34.976864 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":301,"threads_started":1}
I20260928 14:53:34.978605 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:34.963604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.14.241.254:38925)
0928 14:53:34.964639 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:34.964643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:34.964658 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:53:34.964701 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:34.964705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:34.964772 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:34.964866 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:34.966647 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.967406 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.969254 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.969793 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.969911 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:34.972500 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:34.972506 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:34.972511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:34.972568 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:34.975639 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:34.976497 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:34.976503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:34.976506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:34.976631 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:34.977232 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:34.977236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:34.977238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:34.977488 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:34.977662 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:34.978043 (+ 381us) server_negotiation.cc:299] Negotiation successful
0928 14:53:34.978231 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":944,"thread_start_us":829,"threads_started":1}
I20260928 14:53:34.979532 30193 heartbeater.cc:347] Connected to a master server at 127.14.241.254:38925
I20260928 14:53:34.979879 30193 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:34.980707 30193 heartbeater.cc:511] Master 127.14.241.254:38925 requested a full tablet report, sending...
I20260928 14:53:34.982158 29743 ts_manager.cc:194] Registered new tserver with Master: f08758e4b30c4b4b91400a9d9f8408c0 (127.14.241.195:38041)
I20260928 14:53:34.983002 29743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:37561
I20260928 14:53:34.993484 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:53:35.000151 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:34.994339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49360 (local address 127.14.241.254:38925)
0928 14:53:34.994489 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:34.994494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:34.994908 (+ 414us) server_negotiation.cc:407] Connection header received
0928 14:53:34.995064 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:34.995068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:34.995225 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:34.995365 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:53:34.995702 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.996463 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.998019 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:34.998493 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:34.998583 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:34.999088 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:53:34.999661 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:53:34.999784 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:34.999872 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 14:53:34.999940 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:53:35.001098 29743 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:53:35.008678 29743 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:53:35.033717 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.020462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:38041 (local address 127.0.0.1:53068)
0928 14:53:35.022477 (+ 2015us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:35.022496 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:35.022603 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:35.027239 (+ 4636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:35.027244 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:35.027277 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:35.027737 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:35.027749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.029529 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.029535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:35.031216 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:35.031261 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.033018 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.033024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:35.033174 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.033210 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 14:53:35.033296 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 14:53:35.033435 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1896,"mutex_wait_us":23,"thread_start_us":609,"threads_started":1}
I20260928 14:53:35.034610 30206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.020177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:39749 (local address 127.0.0.1:55198)
0928 14:53:35.021487 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:35.021534 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:35.021737 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:35.026906 (+ 5169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:35.026911 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:35.026940 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:35.027396 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:35.027410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.029207 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.029217 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:35.031856 (+ 2639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:35.031872 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.033864 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.033870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:35.034017 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.034046 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 14:53:35.034130 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:53:35.034259 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":214,"thread_start_us":4793,"threads_started":1}
I20260928 14:53:35.034973 30204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.020648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.14.241.195:38041)
0928 14:53:35.026674 (+ 6026us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:35.026705 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:35.026737 (+ 32us) server_negotiation.cc:407] Connection header received
0928 14:53:35.026830 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:35.026836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:35.027036 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:35.027226 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:35.027925 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.029412 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.031447 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.032807 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.034261 (+ 1454us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.034437 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:35.034507 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 14:53:35.034669 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5863,"thread_start_us":5712,"threads_started":1}
I20260928 14:53:35.035346 30203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.020606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55198 (local address 127.14.241.194:39749)
0928 14:53:35.026280 (+ 5674us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:35.026290 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:35.026326 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:53:35.026431 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:35.026438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:35.026700 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:35.026891 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:35.027586 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.029048 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.032238 (+ 3190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.033626 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.033789 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.033960 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:35.034953 (+ 993us) server_negotiation.cc:299] Negotiation successful
0928 14:53:35.035129 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5498,"thread_start_us":5390,"threads_started":1}
I20260928 14:53:35.038666 29997 tablet_service.cc:1467] Processing CreateTablet for tablet 6c40481cd9c14dc793f92bab4c0df7e7 (DEFAULT_TABLE table=test-table [id=b61ed86bc89f40a4b737fa527da14876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:35.039778 29997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c40481cd9c14dc793f92bab4c0df7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:35.041533 30205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.021964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:45341 (local address 127.0.0.1:59602)
0928 14:53:35.025263 (+ 3299us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:35.025353 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:35.025605 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:35.026713 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:35.026721 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:35.026753 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:35.027331 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:35.027345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.029161 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.029165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:35.031864 (+ 2699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:35.031874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.040676 (+ 8802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.040682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:35.040870 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.040905 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:53:35.041005 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 14:53:35.041145 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3176,"mutex_wait_us":165,"thread_start_us":1541,"threads_started":1}
I20260928 14:53:35.041554 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.022491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59602 (local address 127.14.241.193:45341)
0928 14:53:35.024800 (+ 2309us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:35.024811 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:35.025481 (+ 670us) server_negotiation.cc:407] Connection header received
0928 14:53:35.026091 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:35.026101 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:35.026395 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:35.026571 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:35.027551 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.028982 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.032038 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.040369 (+ 8331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.040558 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.040714 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:35.041072 (+ 358us) server_negotiation.cc:299] Negotiation successful
0928 14:53:35.041243 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2105,"thread_start_us":1974,"threads_started":1}
I20260928 14:53:35.041779 30128 tablet_service.cc:1467] Processing CreateTablet for tablet 6c40481cd9c14dc793f92bab4c0df7e7 (DEFAULT_TABLE table=test-table [id=b61ed86bc89f40a4b737fa527da14876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:35.043123 30128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c40481cd9c14dc793f92bab4c0df7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:35.045205 29865 tablet_service.cc:1467] Processing CreateTablet for tablet 6c40481cd9c14dc793f92bab4c0df7e7 (DEFAULT_TABLE table=test-table [id=b61ed86bc89f40a4b737fa527da14876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:35.046633 29865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c40481cd9c14dc793f92bab4c0df7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:35.054541 30209 tablet_bootstrap.cc:492] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: Bootstrap starting.
I20260928 14:53:35.056610 30210 tablet_bootstrap.cc:492] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: Bootstrap starting.
I20260928 14:53:35.059935 30209 tablet_bootstrap.cc:654] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:35.059935 30210 tablet_bootstrap.cc:654] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:35.061252 30211 tablet_bootstrap.cc:492] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: Bootstrap starting.
I20260928 14:53:35.061277 30210 log.cc:824] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:35.061277 30209 log.cc:824] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:35.064483 30211 tablet_bootstrap.cc:654] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:35.066135 30211 log.cc:824] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:35.067759 30209 tablet_bootstrap.cc:492] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: No bootstrap required, opened a new log
I20260928 14:53:35.068061 30209 ts_tablet_manager.cc:1397] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.008s
I20260928 14:53:35.068885 30210 tablet_bootstrap.cc:492] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: No bootstrap required, opened a new log
I20260928 14:53:35.069120 30210 ts_tablet_manager.cc:1397] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260928 14:53:35.069120 30211 tablet_bootstrap.cc:492] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: No bootstrap required, opened a new log
I20260928 14:53:35.069380 30211 ts_tablet_manager.cc:1397] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260928 14:53:35.071645 29930 heartbeater.cc:503] Master 127.14.241.254:38925 was elected leader, sending a full tablet report...
I20260928 14:53:35.078203 30210 raft_consensus.cc:348] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } }
I20260928 14:53:35.078609 30210 raft_consensus.cc:374] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:35.078722 30210 raft_consensus.cc:729] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118b5b082c6f484493a5cc2ce461a504, State: Initialized, Role: FOLLOWER
I20260928 14:53:35.079305 30210 consensus_queue.cc:260] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } }
I20260928 14:53:35.079481 30209 raft_consensus.cc:348] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } }
I20260928 14:53:35.079639 30211 raft_consensus.cc:348] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } }
I20260928 14:53:35.080004 30209 raft_consensus.cc:374] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:35.080021 30211 raft_consensus.cc:374] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:35.080106 30209 raft_consensus.cc:729] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08758e4b30c4b4b91400a9d9f8408c0, State: Initialized, Role: FOLLOWER
I20260928 14:53:35.080111 30211 raft_consensus.cc:729] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f503ec409e142f9a31c44b82c3d538c, State: Initialized, Role: FOLLOWER
I20260928 14:53:35.080796 30209 consensus_queue.cc:260] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } }
I20260928 14:53:35.080997 30062 heartbeater.cc:503] Master 127.14.241.254:38925 was elected leader, sending a full tablet report...
I20260928 14:53:35.081696 30210 ts_tablet_manager.cc:1428] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260928 14:53:35.081875 30193 heartbeater.cc:503] Master 127.14.241.254:38925 was elected leader, sending a full tablet report...
I20260928 14:53:35.082232 30211 consensus_queue.cc:260] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } }
I20260928 14:53:35.082592 30209 ts_tablet_manager.cc:1428] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: Time spent starting tablet: real 0.014s user 0.008s sys 0.006s
I20260928 14:53:35.084151 30211 ts_tablet_manager.cc:1428] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: Time spent starting tablet: real 0.015s user 0.007s sys 0.006s
W20260928 14:53:35.206477 30194 tablet.cc:2369] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:53:35.250955 30063 tablet.cc:2369] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:53:35.290843 29931 tablet.cc:2369] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:53:35.525746 30215 raft_consensus.cc:482] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:53:35.526052 30215 raft_consensus.cc:504] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } }
I20260928 14:53:35.527720 30215 leader_election.cc:288] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f08758e4b30c4b4b91400a9d9f8408c0 (127.14.241.195:38041), 8f503ec409e142f9a31c44b82c3d538c (127.14.241.193:45341)
I20260928 14:53:35.534665 30204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.528096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:49683 (local address 127.14.241.195:38041)
0928 14:53:35.528338 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:35.528345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:35.528707 (+ 362us) server_negotiation.cc:407] Connection header received
0928 14:53:35.528928 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:35.528935 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:35.529015 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:35.529102 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:35.529752 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.530634 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.532701 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.533659 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.533778 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.533834 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:35.534273 (+ 439us) server_negotiation.cc:299] Negotiation successful
0928 14:53:35.534408 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 14:53:35.534700 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.528007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:38041 (local address 127.14.241.194:49683)
0928 14:53:35.528586 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:35.528602 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:35.528760 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:35.529213 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:35.529218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:35.529252 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:35.529602 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:35.529615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.530752 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.530757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:35.532556 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:35.532568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.533862 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.533869 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:35.534056 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.534091 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:53:35.534188 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 14:53:35.534337 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":318,"threads_started":1}
I20260928 14:53:35.535108 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.529021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:47933 (local address 127.14.241.193:45341)
0928 14:53:35.529217 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:35.529223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:35.529375 (+ 152us) server_negotiation.cc:407] Connection header received
0928 14:53:35.529529 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:35.529534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:35.529610 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:35.529707 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:35.530280 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.531257 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.533162 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.534102 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.534263 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.534328 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:35.534751 (+ 423us) server_negotiation.cc:299] Negotiation successful
0928 14:53:35.534890 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 14:53:35.535117 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.528710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:45341 (local address 127.14.241.194:47933)
0928 14:53:35.529283 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:35.529298 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:35.529412 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:35.529735 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:35.529739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:35.529762 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:35.530146 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:35.530158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.531370 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.531374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:35.533000 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:35.533015 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.534365 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.534371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:35.534553 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.534585 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 14:53:35.534681 (+ 96us) client_negotiation.cc:240] Negotiation successful
0928 14:53:35.534821 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":280,"threads_started":1}
I20260928 14:53:35.535950 30148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c40481cd9c14dc793f92bab4c0df7e7" candidate_uuid: "118b5b082c6f484493a5cc2ce461a504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" is_pre_election: true
I20260928 14:53:35.536024 29885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c40481cd9c14dc793f92bab4c0df7e7" candidate_uuid: "118b5b082c6f484493a5cc2ce461a504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f503ec409e142f9a31c44b82c3d538c" is_pre_election: true
I20260928 14:53:35.536504 30148 raft_consensus.cc:2454] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 118b5b082c6f484493a5cc2ce461a504 in term 0.
I20260928 14:53:35.536530 29885 raft_consensus.cc:2454] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 118b5b082c6f484493a5cc2ce461a504 in term 0.
I20260928 14:53:35.537091 29953 leader_election.cc:302] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118b5b082c6f484493a5cc2ce461a504, f08758e4b30c4b4b91400a9d9f8408c0; no voters:
I20260928 14:53:35.537441 30215 raft_consensus.cc:2781] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:53:35.537595 30215 raft_consensus.cc:482] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:53:35.537683 30215 raft_consensus.cc:3037] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:35.540709 30215 raft_consensus.cc:504] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } }
I20260928 14:53:35.541347 30215 leader_election.cc:288] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [CANDIDATE]: Term 1 election: Requested vote from peers f08758e4b30c4b4b91400a9d9f8408c0 (127.14.241.195:38041), 8f503ec409e142f9a31c44b82c3d538c (127.14.241.193:45341)
I20260928 14:53:35.541849 29885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c40481cd9c14dc793f92bab4c0df7e7" candidate_uuid: "118b5b082c6f484493a5cc2ce461a504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f503ec409e142f9a31c44b82c3d538c"
I20260928 14:53:35.541837 30148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c40481cd9c14dc793f92bab4c0df7e7" candidate_uuid: "118b5b082c6f484493a5cc2ce461a504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08758e4b30c4b4b91400a9d9f8408c0"
I20260928 14:53:35.542052 29885 raft_consensus.cc:3037] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:35.542052 30148 raft_consensus.cc:3037] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:35.545167 29885 raft_consensus.cc:2454] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 118b5b082c6f484493a5cc2ce461a504 in term 1.
I20260928 14:53:35.545167 30148 raft_consensus.cc:2454] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 118b5b082c6f484493a5cc2ce461a504 in term 1.
I20260928 14:53:35.545591 29953 leader_election.cc:302] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118b5b082c6f484493a5cc2ce461a504, f08758e4b30c4b4b91400a9d9f8408c0; no voters:
I20260928 14:53:35.545853 30215 raft_consensus.cc:2781] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:53:35.546252 30215 raft_consensus.cc:686] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 1 LEADER]: Becoming Leader. State: Replica: 118b5b082c6f484493a5cc2ce461a504, State: Running, Role: LEADER
I20260928 14:53:35.546846 30215 consensus_queue.cc:237] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } } peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } }
I20260928 14:53:35.551899 29742 catalog_manager.cc:5495] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 118b5b082c6f484493a5cc2ce461a504 (127.14.241.194). New cstate: current_term: 1 leader_uuid: "118b5b082c6f484493a5cc2ce461a504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118b5b082c6f484493a5cc2ce461a504" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 39749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 } health_report { overall_health: UNKNOWN } } }
I20260928 14:53:35.602742 30227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.597181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.14.241.194:39749)
0928 14:53:35.597624 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:35.597629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:35.597648 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:53:35.597861 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:35.597864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:35.597944 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:35.598078 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:53:35.598451 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.599144 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.600580 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.601016 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.601119 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.601618 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:53:35.602203 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:53:35.602331 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:35.602420 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 14:53:35.602488 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":239,"threads_started":1}
I20260928 14:53:35.611125 30148 raft_consensus.cc:1261] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0 [term 1 FOLLOWER]: Refusing update from remote peer 118b5b082c6f484493a5cc2ce461a504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:53:35.611124 29885 raft_consensus.cc:1261] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c [term 1 FOLLOWER]: Refusing update from remote peer 118b5b082c6f484493a5cc2ce461a504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:53:35.612090 30220 consensus_queue.cc:1035] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f503ec409e142f9a31c44b82c3d538c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 45341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:53:35.612481 30215 consensus_queue.cc:1035] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f08758e4b30c4b4b91400a9d9f8408c0" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:53:35.639953 29743 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:53:35.640180 29743 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:53:35.643433 29743 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Sending DeleteTablet for 3 replicas of tablet 6c40481cd9c14dc793f92bab4c0df7e7
I20260928 14:53:35.644567 29997 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c40481cd9c14dc793f92bab4c0df7e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:35 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:55198
I20260928 14:53:35.644848 29865 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c40481cd9c14dc793f92bab4c0df7e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:35 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:59602
I20260928 14:53:35.645048 30128 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c40481cd9c14dc793f92bab4c0df7e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:35 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:53068
I20260928 14:53:35.645588 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 29781
I20260928 14:53:35.660259 30241 tablet_replica.cc:330] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: stopping tablet replica
I20260928 14:53:35.661016 30241 raft_consensus.cc:2229] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:53:35.661736 30241 raft_consensus.cc:2258] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:53:35.664206 30241 ts_tablet_manager.cc:1905] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:53:35.672443 30241 ts_tablet_manager.cc:1918] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:53:35.672724 30241 log.cc:1196] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/6c40481cd9c14dc793f92bab4c0df7e7
I20260928 14:53:35.672732 30242 tablet_replica.cc:330] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: stopping tablet replica
I20260928 14:53:35.673249 30242 raft_consensus.cc:2229] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:53:35.673365 30241 ts_tablet_manager.cc:1939] T 6c40481cd9c14dc793f92bab4c0df7e7 P 8f503ec409e142f9a31c44b82c3d538c: Deleting consensus metadata
I20260928 14:53:35.676002 29731 catalog_manager.cc:4841] TS 8f503ec409e142f9a31c44b82c3d538c (127.14.241.193:45341): tablet 6c40481cd9c14dc793f92bab4c0df7e7 (table test-table [id=b61ed86bc89f40a4b737fa527da14876]) successfully deleted
I20260928 14:53:35.676844 30242 raft_consensus.cc:2258] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:53:35.677902 30240 tablet_replica.cc:330] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: stopping tablet replica
I20260928 14:53:35.678432 30240 raft_consensus.cc:2229] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:53:35.679132 30242 ts_tablet_manager.cc:1905] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:53:35.679272 30240 raft_consensus.cc:2258] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:53:35.681182 30240 ts_tablet_manager.cc:1905] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:53:35.685984 30242 ts_tablet_manager.cc:1918] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:53:35.686195 30242 log.cc:1196] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/6c40481cd9c14dc793f92bab4c0df7e7
I20260928 14:53:35.686744 30242 ts_tablet_manager.cc:1939] T 6c40481cd9c14dc793f92bab4c0df7e7 P f08758e4b30c4b4b91400a9d9f8408c0: Deleting consensus metadata
I20260928 14:53:35.688637 30240 ts_tablet_manager.cc:1918] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:53:35.688644 29731 catalog_manager.cc:4841] TS f08758e4b30c4b4b91400a9d9f8408c0 (127.14.241.195:38041): tablet 6c40481cd9c14dc793f92bab4c0df7e7 (table test-table [id=b61ed86bc89f40a4b737fa527da14876]) successfully deleted
I20260928 14:53:35.688872 30240 log.cc:1196] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/6c40481cd9c14dc793f92bab4c0df7e7
I20260928 14:53:35.689463 30240 ts_tablet_manager.cc:1939] T 6c40481cd9c14dc793f92bab4c0df7e7 P 118b5b082c6f484493a5cc2ce461a504: Deleting consensus metadata
I20260928 14:53:35.689834 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.646808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59612 (local address 127.14.241.193:45341)
0928 14:53:35.646984 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:35.646990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:35.669381 (+ 22391us) server_negotiation.cc:407] Connection header received
0928 14:53:35.669917 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:35.669924 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:35.670018 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:35.670094 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:35.673811 (+ 3717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.675161 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.677262 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.677856 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.677959 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.681711 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:35.681735 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:35.681755 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:35.681808 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:35.685795 (+ 3987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:35.686847 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:35.686857 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:35.686860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:35.687031 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:35.687517 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:35.687522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:35.687525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:35.687997 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:35.688331 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:35.689223 (+ 892us) server_negotiation.cc:299] Negotiation successful
0928 14:53:35.689484 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:53:35.690640 29925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:35.691231 29728 catalog_manager.cc:4841] TS 118b5b082c6f484493a5cc2ce461a504 (127.14.241.194:39749): tablet 6c40481cd9c14dc793f92bab4c0df7e7 (table test-table [id=b61ed86bc89f40a4b737fa527da14876]) successfully deleted
I20260928 14:53:35.780117 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 29781
I20260928 14:53:35.795778 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 29934
I20260928 14:53:35.811372 30227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.796509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55224 (local address 127.14.241.194:39749)
0928 14:53:35.796726 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:35.796730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:35.796748 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:53:35.796970 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:35.796974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:35.797039 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:35.797127 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:35.798814 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.799614 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.801514 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.801916 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.802005 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.805272 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:35.805288 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:35.805304 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:35.805340 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:35.808447 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:35.809362 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:35.809369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:35.809371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:35.809504 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:35.809887 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:35.809890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:35.809892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:35.810251 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:35.810488 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:35.810806 (+ 318us) server_negotiation.cc:299] Negotiation successful
0928 14:53:35.811005 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260928 14:53:35.812108 30057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:35.927206 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 29934
I20260928 14:53:35.942981 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 30065
I20260928 14:53:35.956459 30204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:35.943761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53072 (local address 127.14.241.195:38041)
0928 14:53:35.943944 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:35.943950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:35.943970 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:53:35.944098 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:35.944102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:35.944166 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:35.944244 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:35.945585 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.946343 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.947624 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:35.948024 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:35.948129 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:35.950634 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:35.950649 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:35.950664 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:35.950702 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:35.953621 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:35.954337 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:35.954345 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:35.954348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:35.954527 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:35.954878 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:35.954882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:35.954886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:35.955359 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:35.955661 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:35.955832 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 14:53:35.956060 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 14:53:35.957226 30188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:36.045492 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 30065
I20260928 14:53:36.059441 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 29711
I20260928 14:53:36.060598 29774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:36.177503 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 29711
I20260928 14:53:36.196480 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38925
--webserver_interface=127.14.241.254
--webserver_port=37827
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:38925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:36.519634 30248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:36.519934 30248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:36.520066 30248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:36.527427 30248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:53:36.527536 30248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:36.527606 30248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:36.527686 30248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:53:36.527743 30248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:53:36.537766 30248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:38925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=37827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:36.539088 30248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:36.540510 30248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:36.551681 30253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:36.551600 30254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:36.551617 30256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:36.552456 30248 server_base.cc:1034] running on GCE node
I20260928 14:53:36.553289 30248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:36.554739 30248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:36.555958 30248 hybrid_clock.cc:648] HybridClock initialized: now 1790607216555902 us; error 45 us; skew 500 ppm
I20260928 14:53:36.556350 30248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:36 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607216, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:36.559407 30248 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:53:36.560990 30248 webserver.cc:460] Webserver started at http://127.14.241.254:37827/ using document root <none> and password file <none>
I20260928 14:53:36.561674 30248 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:36.561802 30248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:36.567459 30248 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260928 14:53:36.570485 30263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:36.571911 30248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:53:36.572083 30248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "6344471ac5fd4c6bb2e0e300525ba823"
format_stamp: "Formatted at 2026-09-28 14:53:30 on dist-test-slave-vk40"
server_key: "528c0f03898e3b5322ca8303892e2bc3"
server_key_iv: "63611d91a577b0da8bee6db70f6e51c4"
server_key_version: "encryptionkey@0"
I20260928 14:53:36.573196 30248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:36.588047 30248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:36.592199 30248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:36.592492 30248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:36.609790 30248 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:38925
I20260928 14:53:36.609797 30314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:38925 every 8 connection(s)
I20260928 14:53:36.611300 30248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:53:36.618059 30315 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:53:36.618494 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 30248
I20260928 14:53:36.619246 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:45341
--local_ip_for_outbound_sockets=127.14.241.193
--tserver_master_addrs=127.14.241.254:38925
--webserver_port=40581
--webserver_interface=127.14.241.193
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:53:36.621400 30315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Bootstrap starting.
I20260928 14:53:36.646961 30315 log.cc:824] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:36.668222 30315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:53:36.668926 30315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823: Bootstrap complete.
I20260928 14:53:36.682451 30315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:36.683871 30315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6344471ac5fd4c6bb2e0e300525ba823, State: Initialized, Role: FOLLOWER
I20260928 14:53:36.684715 30315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:36.685032 30315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:53:36.685168 30315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:53:36.685365 30315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 2 FOLLOWER]: Advancing to term 3
I20260928 14:53:36.691114 30315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:36.691731 30315 leader_election.cc:302] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6344471ac5fd4c6bb2e0e300525ba823; no voters:
I20260928 14:53:36.692425 30315 leader_election.cc:288] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 14:53:36.692559 30319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 3 FOLLOWER]: Leader election won for term 3
I20260928 14:53:36.693645 30319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [term 3 LEADER]: Becoming Leader. State: Replica: 6344471ac5fd4c6bb2e0e300525ba823, State: Running, Role: LEADER
I20260928 14:53:36.694180 30319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } }
I20260928 14:53:36.694499 30315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:53:36.696522 30321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6344471ac5fd4c6bb2e0e300525ba823. Latest consensus state: current_term: 3 leader_uuid: "6344471ac5fd4c6bb2e0e300525ba823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } } }
I20260928 14:53:36.696998 30321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:36.696547 30320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6344471ac5fd4c6bb2e0e300525ba823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6344471ac5fd4c6bb2e0e300525ba823" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38925 } } }
I20260928 14:53:36.697526 30320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6344471ac5fd4c6bb2e0e300525ba823 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:36.706239 30327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:53:36.714401 30327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:53:36.715965 30327 catalog_manager.cc:1259] Loaded cluster ID: a8a85e0da3a34202a555421e0e577685
I20260928 14:53:36.716122 30327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260928 14:53:36.718153 30327 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f41b0fd19ad google::LogMessage::Fail() at ??:0
@ 0x7f41b0fd488a google::LogMessage::SendToLog() at ??:0
@ 0x7f41b0fd14de google::LogMessage::Flush() at ??:0
@ 0x7f41b0fd2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f41c6a04512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f41c6a01976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
I20260928 14:53:36.739034 30337 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f41c6a89610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f41c6a89439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f41b3c1d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f41b3c2091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f41b3c20799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f41b3bfbeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f41b6ed56db start_thread at ??:0
@ 0x7f41afe6b61f clone at ??:0
W20260928 14:53:37.008194 30317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:37.008522 30317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:37.008656 30317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:37.015905 30317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:37.016026 30317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:37.016173 30317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:53:37.026041 30317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:45341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=40581
--tserver_master_addrs=127.14.241.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:37.027380 30317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:37.028797 30317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:37.041571 30356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:37.041625 30359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:37.041543 30357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:37.042729 30317 server_base.cc:1034] running on GCE node
I20260928 14:53:37.043370 30317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:37.044353 30317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:37.045605 30317 hybrid_clock.cc:648] HybridClock initialized: now 1790607217045539 us; error 56 us; skew 500 ppm
I20260928 14:53:37.045948 30317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:37 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607217, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:37.049155 30317 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:53:37.050591 30317 webserver.cc:460] Webserver started at http://127.14.241.193:40581/ using document root <none> and password file <none>
I20260928 14:53:37.051254 30317 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:37.051373 30317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:37.056955 30317 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260928 14:53:37.059791 30366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:37.061224 30317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:53:37.061450 30317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "8f503ec409e142f9a31c44b82c3d538c"
format_stamp: "Formatted at 2026-09-28 14:53:31 on dist-test-slave-vk40"
server_key: "a087d588a2ba12a478eefd5f821ca959"
server_key_iv: "defaa60bf0dd1872b3c12862ddfff986"
server_key_version: "encryptionkey@0"
I20260928 14:53:37.062475 30317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:37.077816 30317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:37.082070 30317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:37.082356 30317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:37.083382 30317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:37.084781 30317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:37.084878 30317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:37.084996 30317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:37.085065 30317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:37.118319 30317 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:45341
I20260928 14:53:37.118371 30478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:45341 every 8 connection(s)
I20260928 14:53:37.119726 30317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:53:37.124545 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:37.122694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:32917)
0928 14:53:37.123430 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:37.123646 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":288,"threads_started":1}
W20260928 14:53:37.125169 30479 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:38925 (0 consecutive failures): Network error: Failed to ping master at 127.14.241.254:38925: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
I20260928 14:53:37.125982 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:37.125666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:42077)
0928 14:53:37.125816 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:37.125883 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 14:53:37.126245 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 30317
I20260928 14:53:37.126802 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:37.126481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:50599)
0928 14:53:37.126607 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:37.126699 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 14:53:37.126835 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:39749
--local_ip_for_outbound_sockets=127.14.241.194
--tserver_master_addrs=127.14.241.254:38925
--webserver_port=39137
--webserver_interface=127.14.241.194
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:37.127008 30479 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:53:37.453715 30482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:37.453984 30482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:37.454079 30482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:37.461232 30482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:37.461360 30482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:37.461484 30482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:53:37.471318 30482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:39749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=39137
--tserver_master_addrs=127.14.241.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:37.472621 30482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:37.473953 30482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:37.486722 30487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:37.486745 30490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:37.486665 30488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:37.487543 30482 server_base.cc:1034] running on GCE node
I20260928 14:53:37.488090 30482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:37.488997 30482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:37.490187 30482 hybrid_clock.cc:648] HybridClock initialized: now 1790607217490145 us; error 35 us; skew 500 ppm
I20260928 14:53:37.490475 30482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:37 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607217, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:37.493417 30482 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:53:37.494757 30482 webserver.cc:460] Webserver started at http://127.14.241.194:39137/ using document root <none> and password file <none>
I20260928 14:53:37.495407 30482 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:37.495515 30482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:37.500717 30482 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:53:37.503278 30497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:37.504603 30482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:53:37.504750 30482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "118b5b082c6f484493a5cc2ce461a504"
format_stamp: "Formatted at 2026-09-28 14:53:31 on dist-test-slave-vk40"
server_key: "155a2fc84a437fdd9baec48e56ef2aae"
server_key_iv: "b92431cb34676643049876ecd8ea4701"
server_key_version: "encryptionkey@0"
I20260928 14:53:37.505697 30482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:37.531698 30482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:37.535709 30482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:37.535977 30482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:37.537005 30482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:37.538383 30482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:37.538477 30482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:37.538583 30482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:37.538651 30482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:37.569563 30482 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:39749
I20260928 14:53:37.569633 30609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:39749 every 8 connection(s)
I20260928 14:53:37.571007 30482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:53:37.575776 30612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:37.573977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:52515)
0928 14:53:37.574669 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:37.574895 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":266,"threads_started":1}
W20260928 14:53:37.576382 30610 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:38925 (0 consecutive failures): Network error: Failed to ping master at 127.14.241.254:38925: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
I20260928 14:53:37.577224 30612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:37.576895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:39911)
0928 14:53:37.577046 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:37.577119 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 14:53:37.577987 30612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:37.577683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:39695)
0928 14:53:37.577816 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:37.577887 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260928 14:53:37.578212 30610 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:53:37.579313 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 30482
I20260928 14:53:37.579865 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:38041
--local_ip_for_outbound_sockets=127.14.241.195
--tserver_master_addrs=127.14.241.254:38925
--webserver_port=38693
--webserver_interface=127.14.241.195
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:37.901834 30613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:37.902129 30613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:37.902257 30613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:37.909507 30613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:37.909633 30613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:37.909792 30613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:53:37.919574 30613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:38041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=38693
--tserver_master_addrs=127.14.241.254:38925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:37.920934 30613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:37.922272 30613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:37.934974 30619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:37.935035 30618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:37.934998 30621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:37.936100 30613 server_base.cc:1034] running on GCE node
I20260928 14:53:37.936728 30613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:37.937703 30613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:37.938900 30613 hybrid_clock.cc:648] HybridClock initialized: now 1790607217938843 us; error 48 us; skew 500 ppm
I20260928 14:53:37.939219 30613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:37 dist-test-slave-vk40 krb5kdc[29159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607217, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:37.942348 30613 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:53:37.943778 30613 webserver.cc:460] Webserver started at http://127.14.241.195:38693/ using document root <none> and password file <none>
I20260928 14:53:37.944504 30613 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:37.944615 30613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:37.949934 30613 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:53:37.952600 30628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:37.954084 30613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:53:37.954241 30613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "f08758e4b30c4b4b91400a9d9f8408c0"
format_stamp: "Formatted at 2026-09-28 14:53:32 on dist-test-slave-vk40"
server_key: "23cd034ac9a03ecb931f48e0d0ac63c2"
server_key_iv: "ad9f3126e5bdbac3ef8cf7017e774ef6"
server_key_version: "encryptionkey@0"
I20260928 14:53:37.955214 30613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:37.982918 30613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:37.987336 30613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:37.987577 30613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:37.988596 30613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:37.989949 30613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:37.990041 30613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:37.990123 30613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:37.990175 30613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:38.020356 30613 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:38041
I20260928 14:53:38.020432 30740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:38041 every 8 connection(s)
I20260928 14:53:38.021888 30613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:53:38.026824 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:38.024905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:40925)
0928 14:53:38.025673 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:38.025922 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":304,"threads_started":1}
W20260928 14:53:38.027500 30741 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:38925 (0 consecutive failures): Network error: Failed to ping master at 127.14.241.254:38925: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
I20260928 14:53:38.028358 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:38.027995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:53331)
0928 14:53:38.028132 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:38.028200 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 14:53:38.029138 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:38.028848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:46951)
0928 14:53:38.028957 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:38.029030 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 14:53:38.029351 30741 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:53:38.031492 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 30613
I20260928 14:53:38.128432 30745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:38.127576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:60697)
0928 14:53:38.128029 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:38.128161 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":219,"threads_started":1}
I20260928 14:53:38.579605 30746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:38.578748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:37009)
0928 14:53:38.579257 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:38.579378 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":238,"threads_started":1}
I20260928 14:53:39.030692 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:39.029896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:58359)
0928 14:53:39.030345 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:39.030459 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":207,"threads_started":1}
I20260928 14:53:39.129948 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:39.129212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:36537)
0928 14:53:39.129693 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:39.129782 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":243,"threads_started":1}
I20260928 14:53:39.581343 30749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:39.580576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:36181)
0928 14:53:39.581079 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:39.581165 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":250,"threads_started":1}
I20260928 14:53:40.032272 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:40.031590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:53971)
0928 14:53:40.032029 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:40.032108 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":209,"threads_started":1}
I20260928 14:53:40.131721 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:40.130941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:47539)
0928 14:53:40.131437 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:40.131523 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":270,"threads_started":1}
I20260928 14:53:40.583050 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:40.582325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:33299)
0928 14:53:40.582779 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:40.582865 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":240,"threads_started":1}
I20260928 14:53:41.033946 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:41.033176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:43453)
0928 14:53:41.033640 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:41.033759 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":242,"threads_started":1}
I20260928 14:53:41.133502 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:41.132659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:57809)
0928 14:53:41.133181 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:41.133289 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":244,"threads_started":1}
I20260928 14:53:41.584913 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:41.584064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:37979)
0928 14:53:41.584616 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:41.584709 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":306,"threads_started":1}
I20260928 14:53:42.035735 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:42.034946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:34973)
0928 14:53:42.035437 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:42.035535 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":242,"threads_started":1}
I20260928 14:53:42.135334 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:42.134549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:38161)
0928 14:53:42.135070 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:42.135164 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":266,"threads_started":1}
I20260928 14:53:42.586712 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:42.585878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:34807)
0928 14:53:42.586429 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:42.586522 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":277,"threads_started":1}
I20260928 14:53:43.037400 30759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:43.036675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:60527)
0928 14:53:43.037148 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:43.037229 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":250,"threads_started":1}
I20260928 14:53:43.137075 30760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:43.136249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:59993)
0928 14:53:43.136784 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:43.136903 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":277,"threads_started":1}
I20260928 14:53:43.588522 30761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:43.587722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:41811)
0928 14:53:43.588198 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:43.588343 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":251,"threads_started":1}
I20260928 14:53:44.039021 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:44.038298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:35289)
0928 14:53:44.038752 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:44.038850 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":216,"threads_started":1}
I20260928 14:53:44.139070 30763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:44.138106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:48883)
0928 14:53:44.138708 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:44.138815 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":270,"threads_started":1}
I20260928 14:53:44.590219 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:44.589428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:38283)
0928 14:53:44.589913 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:44.589997 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":203,"threads_started":1}
I20260928 14:53:45.040863 30765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:45.039983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:54975)
0928 14:53:45.040557 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:45.040646 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":280,"threads_started":1}
I20260928 14:53:45.140957 30766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:45.140164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:55657)
0928 14:53:45.140683 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:45.140784 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":286,"threads_started":1}
I20260928 14:53:45.591856 30767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:45.591115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:51093)
0928 14:53:45.591592 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:45.591680 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":246,"threads_started":1}
I20260928 14:53:46.042548 30768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:46.041779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:58341)
0928 14:53:46.042298 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:46.042389 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":263,"threads_started":1}
I20260928 14:53:46.142740 30769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:46.141910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:37151)
0928 14:53:46.142455 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:46.142550 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":252,"threads_started":1}
I20260928 14:53:46.593523 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:46.592761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:32955)
0928 14:53:46.593241 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:46.593329 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":221,"threads_started":1}
I20260928 14:53:47.044263 30771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:47.043484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:44575)
0928 14:53:47.044000 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:47.044089 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":236,"threads_started":1}
I20260928 14:53:47.144443 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:47.143703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:55497)
0928 14:53:47.144128 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:47.144231 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":228,"threads_started":1}
I20260928 14:53:47.595144 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:47.594459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:40219)
0928 14:53:47.594902 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:47.594984 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":249,"threads_started":1}
I20260928 14:53:48.045987 30774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:48.045230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:49601)
0928 14:53:48.045705 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:48.045817 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":242,"threads_started":1}
I20260928 14:53:48.146163 30775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:48.145401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:52801)
0928 14:53:48.145881 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:48.145981 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":230,"threads_started":1}
I20260928 14:53:48.596786 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:48.596031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:55799)
0928 14:53:48.596500 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:48.596600 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":207,"threads_started":1}
I20260928 14:53:49.047621 30777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:49.046909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:52173)
0928 14:53:49.047337 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:49.047434 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":208,"threads_started":1}
I20260928 14:53:49.147929 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:49.147172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:52739)
0928 14:53:49.147652 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:49.147770 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":246,"threads_started":1}
I20260928 14:53:49.598464 30779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:49.597755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:42901)
0928 14:53:49.598187 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:49.598278 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":209,"threads_started":1}
I20260928 14:53:50.049237 30780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:50.048529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:50827)
0928 14:53:50.048992 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:50.049078 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":218,"threads_started":1}
I20260928 14:53:50.149654 30781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:50.148876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:49799)
0928 14:53:50.149365 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:50.149463 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":226,"threads_started":1}
I20260928 14:53:50.600333 30782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:50.599476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:60755)
0928 14:53:50.599993 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:50.600093 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":236,"threads_started":1}
I20260928 14:53:51.050937 30783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:51.050124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:52007)
0928 14:53:51.050619 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:51.050715 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":248,"threads_started":1}
I20260928 14:53:51.151281 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:51.150505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:49791)
0928 14:53:51.151000 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:51.151089 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":279,"threads_started":1}
I20260928 14:53:51.601948 30785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:51.601183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:37887)
0928 14:53:51.601650 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:51.601752 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":234,"threads_started":1}
I20260928 14:53:52.052670 30786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:52.051848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:37269)
0928 14:53:52.052370 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:52.052475 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":226,"threads_started":1}
I20260928 14:53:52.153234 30787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:52.152500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.193:40979)
0928 14:53:52.152940 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:52.153044 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":221,"threads_started":1}
I20260928 14:53:52.604012 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:52.603169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.194:38573)
0928 14:53:52.603685 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:52.603823 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":255,"threads_started":1}
I20260928 14:53:53.042104 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 30317
I20260928 14:53:53.054616 30790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:53.053779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38925 (local address 127.14.241.195:38593)
0928 14:53:53.054253 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:53.054384 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":222,"threads_started":1}
I20260928 14:53:53.060492 30789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:53.043348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48714 (local address 127.14.241.193:45341)
0928 14:53:53.044234 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:53.044257 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:53.044332 (+ 75us) server_negotiation.cc:407] Connection header received
0928 14:53:53.046337 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:53.046348 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:53.046800 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:53.047100 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:53.048557 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:53.049800 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:53.051008 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:53.051657 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:53.051787 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:53.054606 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:53.054618 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:53.054641 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:53.054697 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:53.057443 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:53.058114 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:53.058121 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:53.058129 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:53.058323 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:53.058616 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:53.058619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:53.058621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:53.059214 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:53.059470 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:53.059795 (+ 325us) server_negotiation.cc:299] Negotiation successful
0928 14:53:53.060160 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":564,"threads_started":1}
I20260928 14:53:53.061849 30474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:53.148598 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 30317
I20260928 14:53:53.163321 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 30482
I20260928 14:53:53.180676 30792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:53.164206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50752 (local address 127.14.241.194:39749)
0928 14:53:53.164767 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:53.164782 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:53.164827 (+ 45us) server_negotiation.cc:407] Connection header received
0928 14:53:53.166359 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:53.166370 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:53.166834 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:53.167173 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:53.168744 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:53.169982 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:53.171238 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:53.171923 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:53.172099 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:53.174670 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:53.174678 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:53.174695 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:53.174734 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:53.177455 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:53.178096 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:53.178103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:53.178110 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:53.178279 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:53.178656 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:53.178659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:53.178661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:53.179184 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:53.179479 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:53.179944 (+ 465us) server_negotiation.cc:299] Negotiation successful
0928 14:53:53.180386 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":256,"threads_started":1}
I20260928 14:53:53.182101 30605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:53.265688 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 30482
I20260928 14:53:53.280766 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 30613
I20260928 14:53:53.298398 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:53.281683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.14.241.195:38041)
0928 14:53:53.282288 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:53.282302 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:53.282344 (+ 42us) server_negotiation.cc:407] Connection header received
0928 14:53:53.283916 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:53.283927 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:53.284439 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:53.284734 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:53.286049 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:53.287319 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:53.288692 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:53.289419 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:53.289585 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:53.292056 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:53.292064 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:53.292080 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:53.292119 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:53.295076 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:53.295706 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:53.295712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:53.295719 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:53.295909 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:53.296209 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:53.296212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:53.296214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:53.296825 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:53.297074 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:53.297556 (+ 482us) server_negotiation.cc:299] Negotiation successful
0928 14:53:53.298052 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":296,"threads_started":1}
I20260928 14:53:53.299844 30736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:53.383656 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 30613
2026-09-28T14:53:53Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24344 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:53:53 dist-test-slave-vk40 krb5kdc[30798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:53:53 dist-test-slave-vk40 krb5kdc[30798](info): set up 2 sockets
Sep 28 14:53:53 dist-test-slave-vk40 krb5kdc[30798](info): commencing operation
krb5kdc: starting...
W20260928 14:53:54.399412 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:53:54 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:53:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:53:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:53:54.640712 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41753
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:41753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:54.970116 30811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:54.970476 30811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:54.970614 30811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:54.978057 30811 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 14:53:54.978176 30811 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 14:53:54.978230 30811 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 14:53:54.978297 30811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:53:54.978374 30811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:54.978472 30811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:54.978538 30811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:53:54.978617 30811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:53:54.988897 30811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:41753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:54.990223 30811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:54.991572 30811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:55.003698 30816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:55.003737 30819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:55.003762 30817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:55.004101 30811 server_base.cc:1034] running on GCE node
I20260928 14:53:55.004952 30811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:55.006433 30811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:55.007655 30811 hybrid_clock.cc:648] HybridClock initialized: now 1790607235007576 us; error 60 us; skew 500 ppm
I20260928 14:53:55.008005 30811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:55 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607235, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:55.011474 30811 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:53:55.013113 30811 webserver.cc:460] Webserver started at http://127.14.241.254:33077/ using document root <none> and password file <none>
I20260928 14:53:55.013827 30811 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:55.013926 30811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:55.014232 30811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:55.017280 30811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "13ed2a70f062417b8e67e47350f71f64"
format_stamp: "Formatted at 2026-09-28 14:53:55 on dist-test-slave-vk40"
server_key: "be493fa344bc6a2a5e0bc8eab8c7ea12"
server_key_iv: "1a1787c168a92d1b3aac0a8a512f9743"
server_key_version: "encryptionkey@0"
I20260928 14:53:55.018047 30811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "13ed2a70f062417b8e67e47350f71f64"
format_stamp: "Formatted at 2026-09-28 14:53:55 on dist-test-slave-vk40"
server_key: "be493fa344bc6a2a5e0bc8eab8c7ea12"
server_key_iv: "1a1787c168a92d1b3aac0a8a512f9743"
server_key_version: "encryptionkey@0"
I20260928 14:53:55.023491 30811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:53:55.027055 30826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:55.028641 30811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:53:55.028816 30811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "13ed2a70f062417b8e67e47350f71f64"
format_stamp: "Formatted at 2026-09-28 14:53:55 on dist-test-slave-vk40"
server_key: "be493fa344bc6a2a5e0bc8eab8c7ea12"
server_key_iv: "1a1787c168a92d1b3aac0a8a512f9743"
server_key_version: "encryptionkey@0"
I20260928 14:53:55.029042 30811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:55.043706 30811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:55.048193 30811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:55.048516 30811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:55.066562 30811 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:41753
I20260928 14:53:55.066583 30877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:41753 every 8 connection(s)
I20260928 14:53:55.067966 30811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:53:55.071789 30878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:55.073421 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 30811
I20260928 14:53:55.073612 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:53:55.074244 15303 external_mini_cluster.cc:1428] Setting key 946315896e9640007421e2c092edc038
I20260928 14:53:55.085296 30878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Bootstrap starting.
I20260928 14:53:55.089003 30878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Neither blocks nor log segments found. Creating new log.
Sep 28 14:53:55 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:55.090087 30878 log.cc:824] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:55.093648 30878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: No bootstrap required, opened a new log
I20260928 14:53:55.099853 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:55.076305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.14.241.254:41753)
0928 14:53:55.077166 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:55.077183 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:55.077238 (+ 55us) server_negotiation.cc:407] Connection header received
0928 14:53:55.079602 (+ 2364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:55.079632 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:55.080166 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:55.080663 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:55.082077 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:55.084182 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:55.085730 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:55.086757 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:55.086966 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:55.091073 (+ 4107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:55.091081 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:55.091101 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:55.091143 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:55.094957 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:55.095737 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:55.095746 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:55.095755 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:55.095981 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:55.096341 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:55.096346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:55.096350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:55.097144 (+ 794us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:55.097463 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:55.097908 (+ 445us) server_negotiation.cc:299] Negotiation successful
0928 14:53:55.098394 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":355,"threads_started":1}
I20260928 14:53:55.103416 30878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:53:55.103785 30878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:55.103855 30878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ed2a70f062417b8e67e47350f71f64, State: Initialized, Role: FOLLOWER
I20260928 14:53:55.104455 30878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:53:55.104743 30878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:53:55.104903 30878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:53:55.105093 30878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:55.107362 30878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:53:55.107817 30878 leader_election.cc:302] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13ed2a70f062417b8e67e47350f71f64; no voters:
I20260928 14:53:55.108350 30878 leader_election.cc:288] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:53:55.108481 30883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:53:55.109107 30883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 1 LEADER]: Becoming Leader. State: Replica: 13ed2a70f062417b8e67e47350f71f64, State: Running, Role: LEADER
I20260928 14:53:55.109618 30883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:53:55.110366 30878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:53:55.112430 30885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13ed2a70f062417b8e67e47350f71f64. Latest consensus state: current_term: 1 leader_uuid: "13ed2a70f062417b8e67e47350f71f64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } } }
I20260928 14:53:55.113026 30885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:55.114012 30884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13ed2a70f062417b8e67e47350f71f64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } } }
I20260928 14:53:55.121557 30884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: This master's current role is: LEADER
I20260928 14:53:55.122130 30889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:53:55.131366 30889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:53:55.139696 30889 catalog_manager.cc:1347] Generated new cluster ID: 157fb1c8ed0842be842a650bd9510eb4
I20260928 14:53:55.139822 30889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:53:55.162784 30889 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:53:55.164775 30889 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:53:55.183102 30889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Generated new TSK 0
I20260928 14:53:55.183995 30889 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:53:55.252456 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41753
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:55.585858 30904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:55.586161 30904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:55.586289 30904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:55.593708 30904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:55.593843 30904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:55.594013 30904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:53:55.604256 30904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:55.605669 30904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:55.607033 30904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:55.620102 30909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:55.620028 30910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:55.620062 30912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:55.621016 30904 server_base.cc:1034] running on GCE node
I20260928 14:53:55.621726 30904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:55.622725 30904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:55.623924 30904 hybrid_clock.cc:648] HybridClock initialized: now 1790607235623869 us; error 53 us; skew 500 ppm
I20260928 14:53:55.624230 30904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:55 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607235, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:55.627373 30904 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:53:55.628858 30904 webserver.cc:460] Webserver started at http://127.14.241.193:40827/ using document root <none> and password file <none>
I20260928 14:53:55.629557 30904 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:55.629678 30904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:55.629968 30904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:55.633117 30904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "18e111d3231244499188d4f1c7157f72"
format_stamp: "Formatted at 2026-09-28 14:53:55 on dist-test-slave-vk40"
server_key: "f22c61094b62edea6af65a85c90701bb"
server_key_iv: "debce66641de54b55a84cf549b43a363"
server_key_version: "encryptionkey@0"
I20260928 14:53:55.633919 30904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "18e111d3231244499188d4f1c7157f72"
format_stamp: "Formatted at 2026-09-28 14:53:55 on dist-test-slave-vk40"
server_key: "f22c61094b62edea6af65a85c90701bb"
server_key_iv: "debce66641de54b55a84cf549b43a363"
server_key_version: "encryptionkey@0"
I20260928 14:53:55.639178 30904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260928 14:53:55.642964 30919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:55.644429 30904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:53:55.644593 30904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "18e111d3231244499188d4f1c7157f72"
format_stamp: "Formatted at 2026-09-28 14:53:55 on dist-test-slave-vk40"
server_key: "f22c61094b62edea6af65a85c90701bb"
server_key_iv: "debce66641de54b55a84cf549b43a363"
server_key_version: "encryptionkey@0"
I20260928 14:53:55.644779 30904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:55.661599 30904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:55.665668 30904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:55.665946 30904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:55.666930 30904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:55.668318 30904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:55.668396 30904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:55.668514 30904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:55.668566 30904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:55.699237 30904 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:38353
I20260928 14:53:55.699283 31031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:38353 every 8 connection(s)
I20260928 14:53:55.700620 30904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:53:55.708738 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 30904
I20260928 14:53:55.708933 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:53:55.709553 15303 external_mini_cluster.cc:1428] Setting key d8064b236148c7c040dc70afe32d2b91
Sep 28 14:53:55 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607235, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:55.726815 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:55.703384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:37545 (local address 127.14.241.254:41753)
0928 14:53:55.703902 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:55.703910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:55.704954 (+ 1044us) server_negotiation.cc:407] Connection header received
0928 14:53:55.706809 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:55.706814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:55.706885 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:55.706986 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:55.708708 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:55.709535 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:55.711679 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:55.712329 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:55.712457 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:55.716374 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:55.716382 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:55.716387 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:55.716429 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:55.723195 (+ 6766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:55.724308 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:55.724315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:55.724318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:55.724481 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:55.724943 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:55.724948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:55.724950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:55.725283 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:55.725417 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:55.726217 (+ 800us) server_negotiation.cc:299] Negotiation successful
0928 14:53:55.726449 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":226,"threads_started":1}
I20260928 14:53:55.727715 31035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:55.704025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:37545)
0928 14:53:55.704777 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:55.704826 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:55.706515 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:55.707190 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:55.707198 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:55.707741 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:55.708480 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:55.708509 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:55.709736 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:55.709741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:55.711469 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:55.711485 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:55.712686 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:55.712695 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:55.713017 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:55.713814 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:55.713840 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:55.716058 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:55.723430 (+ 7372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:55.723442 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:55.723463 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:55.724136 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:55.724607 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:55.724612 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:55.724615 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:55.724836 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:55.725422 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:55.725429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:55.725836 (+ 407us) client_negotiation.cc:769] Sending connection context
0928 14:53:55.726088 (+ 252us) client_negotiation.cc:240] Negotiation successful
0928 14:53:55.726518 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":296,"threads_started":1}
I20260928 14:53:55.729539 31032 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41753
I20260928 14:53:55.729892 31032 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:55.730785 31032 heartbeater.cc:511] Master 127.14.241.254:41753 requested a full tablet report, sending...
I20260928 14:53:55.733426 30843 ts_manager.cc:194] Registered new tserver with Master: 18e111d3231244499188d4f1c7157f72 (127.14.241.193:38353)
I20260928 14:53:55.734949 30843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:37545
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:53:55.765378 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41753
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:56.086704 31038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:56.086997 31038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:56.087092 31038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:56.094343 31038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:56.094465 31038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:56.094585 31038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:53:56.104382 31038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:56.105726 31038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:56.107067 31038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:56.120326 31044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:56.120254 31043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:56.120219 31046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:56.125957 31038 server_base.cc:1034] running on GCE node
I20260928 14:53:56.126614 31038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:56.127652 31038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:56.128844 31038 hybrid_clock.cc:648] HybridClock initialized: now 1790607236128783 us; error 63 us; skew 500 ppm
I20260928 14:53:56.129158 31038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:56 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607236, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:56.132323 31038 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:53:56.134073 31038 webserver.cc:460] Webserver started at http://127.14.241.194:45853/ using document root <none> and password file <none>
I20260928 14:53:56.134770 31038 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:56.134879 31038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:56.135133 31038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:56.138157 31038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "9f8b64d817e04c659f289970992c6fb2"
format_stamp: "Formatted at 2026-09-28 14:53:56 on dist-test-slave-vk40"
server_key: "0493eaa971846d4020e859ad5a4680a6"
server_key_iv: "474a876e287f8d344864205f07891b25"
server_key_version: "encryptionkey@0"
I20260928 14:53:56.138887 31038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "9f8b64d817e04c659f289970992c6fb2"
format_stamp: "Formatted at 2026-09-28 14:53:56 on dist-test-slave-vk40"
server_key: "0493eaa971846d4020e859ad5a4680a6"
server_key_iv: "474a876e287f8d344864205f07891b25"
server_key_version: "encryptionkey@0"
I20260928 14:53:56.144126 31038 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 14:53:56.147667 31053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:56.149237 31038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:53:56.149405 31038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "9f8b64d817e04c659f289970992c6fb2"
format_stamp: "Formatted at 2026-09-28 14:53:56 on dist-test-slave-vk40"
server_key: "0493eaa971846d4020e859ad5a4680a6"
server_key_iv: "474a876e287f8d344864205f07891b25"
server_key_version: "encryptionkey@0"
I20260928 14:53:56.149596 31038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:56.175700 31038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:56.180097 31038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:56.180383 31038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:56.181329 31038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:56.182654 31038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:56.182729 31038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:56.182830 31038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:56.182876 31038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:56.218653 31038 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:45409
I20260928 14:53:56.218683 31165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:45409 every 8 connection(s)
I20260928 14:53:56.219950 31038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:53:56.227855 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 31038
I20260928 14:53:56.228032 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:53:56.228523 15303 external_mini_cluster.cc:1428] Setting key 2eb9c0835bae476a0ac27387706caa8c
Sep 28 14:53:56 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607236, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:56.240312 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:56.222468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:59365 (local address 127.14.241.254:41753)
0928 14:53:56.222640 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:56.222644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:56.223886 (+ 1242us) server_negotiation.cc:407] Connection header received
0928 14:53:56.225759 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:56.225766 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:56.225860 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:56.225951 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:56.227526 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.228313 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.230048 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.230569 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.230664 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:56.234663 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:56.234669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:56.234672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:56.234705 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:56.237215 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:56.237962 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:56.237968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:56.237971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:56.238108 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:56.238534 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:56.238538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:56.238540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:56.238829 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:56.238970 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:56.239681 (+ 711us) server_negotiation.cc:299] Negotiation successful
0928 14:53:56.239934 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:53:56.241295 31168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:56.222984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:59365)
0928 14:53:56.223721 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:56.223764 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:56.225484 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:56.226194 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:56.226203 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:56.226779 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:56.227358 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:56.227377 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.228456 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.228460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:56.229916 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:56.229932 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.230786 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.230794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:56.231086 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:56.231787 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:56.231815 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:56.234426 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:56.237363 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:56.237373 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:56.237398 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:56.237836 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:56.238215 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:56.238220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:56.238224 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:56.238426 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:56.238963 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:56.238971 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:56.239317 (+ 346us) client_negotiation.cc:769] Sending connection context
0928 14:53:56.239576 (+ 259us) client_negotiation.cc:240] Negotiation successful
0928 14:53:56.240138 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":294,"threads_started":1}
I20260928 14:53:56.242758 31166 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41753
I20260928 14:53:56.243068 31166 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:56.243775 31166 heartbeater.cc:511] Master 127.14.241.254:41753 requested a full tablet report, sending...
I20260928 14:53:56.245309 30843 ts_manager.cc:194] Registered new tserver with Master: 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409)
I20260928 14:53:56.246372 30843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:59365
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:53:56.280840 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41753
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:53:56.607239 31171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:53:56.607535 31171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:53:56.607630 31171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:53:56.615166 31171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:53:56.615290 31171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:53:56.615412 31171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:53:56.625514 31171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:53:56.626842 31171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:53:56.628211 31171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:53:56.641050 31179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:56.641027 31177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:53:56.641211 31176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:53:56.641845 31171 server_base.cc:1034] running on GCE node
I20260928 14:53:56.642441 31171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:53:56.643356 31171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:53:56.644562 31171 hybrid_clock.cc:648] HybridClock initialized: now 1790607236644495 us; error 47 us; skew 500 ppm
I20260928 14:53:56.644878 31171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:53:56 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607236, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:53:56.653885 31171 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:53:56.655412 31171 webserver.cc:460] Webserver started at http://127.14.241.195:45027/ using document root <none> and password file <none>
I20260928 14:53:56.656086 31171 fs_manager.cc:359] Metadata directory not provided
I20260928 14:53:56.656191 31171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:53:56.656538 31171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:53:56.659459 31171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "333776926c6f4b71af88e695049f7d77"
format_stamp: "Formatted at 2026-09-28 14:53:56 on dist-test-slave-vk40"
server_key: "d23479784a0ca640ff16cc6caac562da"
server_key_iv: "3ddd191ea3b67c3e64c7ec9933fe3a0c"
server_key_version: "encryptionkey@0"
I20260928 14:53:56.660183 31171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "333776926c6f4b71af88e695049f7d77"
format_stamp: "Formatted at 2026-09-28 14:53:56 on dist-test-slave-vk40"
server_key: "d23479784a0ca640ff16cc6caac562da"
server_key_iv: "3ddd191ea3b67c3e64c7ec9933fe3a0c"
server_key_version: "encryptionkey@0"
I20260928 14:53:56.665306 31171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:53:56.668890 31186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:56.670279 31171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:53:56.670450 31171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "333776926c6f4b71af88e695049f7d77"
format_stamp: "Formatted at 2026-09-28 14:53:56 on dist-test-slave-vk40"
server_key: "d23479784a0ca640ff16cc6caac562da"
server_key_iv: "3ddd191ea3b67c3e64c7ec9933fe3a0c"
server_key_version: "encryptionkey@0"
I20260928 14:53:56.670641 31171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:53:56.687350 31171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:53:56.691349 31171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:53:56.691632 31171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:53:56.692574 31171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:53:56.693867 31171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:53:56.693946 31171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:56.694067 31171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:53:56.694121 31171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:53:56.725469 31171 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:42573
I20260928 14:53:56.725492 31298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:42573 every 8 connection(s)
I20260928 14:53:56.726745 31171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:53:56.734138 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 31171
I20260928 14:53:56.734344 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:53:56.734822 15303 external_mini_cluster.cc:1428] Setting key f81e535260268c6ad53ce64680ef48f0
I20260928 14:53:56.738572 31032 heartbeater.cc:503] Master 127.14.241.254:41753 was elected leader, sending a full tablet report...
Sep 28 14:53:56 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607236, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:53:56.746189 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:56.729244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:36093 (local address 127.14.241.254:41753)
0928 14:53:56.729434 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:56.729438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:56.730546 (+ 1108us) server_negotiation.cc:407] Connection header received
0928 14:53:56.732316 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:56.732322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:56.732436 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:56.732547 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:56.734384 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.735207 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.736790 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.737211 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.737319 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:56.740849 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:56.740854 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:56.740857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:56.740897 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:56.743233 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:56.743917 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:56.743921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:56.743923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:56.744045 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:56.744500 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:56.744505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:56.744508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:56.744783 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:56.744925 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:56.745626 (+ 701us) server_negotiation.cc:299] Negotiation successful
0928 14:53:56.745847 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 14:53:56.747128 31301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:56.729674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:36093)
0928 14:53:56.730382 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:56.730425 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:56.732048 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:56.732845 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:56.732859 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:56.733589 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:53:56.734174 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:56.734195 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.735370 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.735373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:56.736655 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:56.736668 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.737431 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.737443 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:56.737683 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:56.738269 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:53:56.738292 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:53:56.740649 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:53:56.743366 (+ 2717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:56.743384 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:56.743401 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:56.743800 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:56.744146 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:53:56.744151 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:53:56.744155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:53:56.744401 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:53:56.744902 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:53:56.744911 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:53:56.745262 (+ 351us) client_negotiation.cc:769] Sending connection context
0928 14:53:56.745508 (+ 246us) client_negotiation.cc:240] Negotiation successful
0928 14:53:56.746030 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":267,"threads_started":1}
I20260928 14:53:56.748622 31299 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41753
I20260928 14:53:56.748966 31299 heartbeater.cc:464] Registering TS with master...
I20260928 14:53:56.749748 31299 heartbeater.cc:511] Master 127.14.241.254:41753 requested a full tablet report, sending...
I20260928 14:53:56.751113 30843 ts_manager.cc:194] Registered new tserver with Master: 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573)
I20260928 14:53:56.751931 30843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:36093
I20260928 14:53:56.762353 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:53:56.779153 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:56.766767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.14.241.254:41753)
0928 14:53:56.766983 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:56.766990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:56.767343 (+ 353us) server_negotiation.cc:407] Connection header received
0928 14:53:56.767485 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:56.767489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:56.767553 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:56.767632 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:56.769004 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.769738 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.771090 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.771525 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.771629 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:56.774028 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:56.774033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:56.774036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:56.774071 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:56.776751 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:56.777437 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:56.777440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:56.777443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:56.777571 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:56.777952 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:56.777955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:56.777957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:56.778256 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:56.778447 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:56.778734 (+ 287us) server_negotiation.cc:299] Negotiation successful
0928 14:53:56.778920 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:53:56.783421 30843 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:53:56.790544 30843 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:53:56.827057 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:56.803871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.14.241.193:38353)
0928 14:53:56.817538 (+ 13667us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:56.817548 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:56.818212 (+ 664us) server_negotiation.cc:407] Connection header received
0928 14:53:56.818327 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:56.818335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:56.818655 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:56.819592 (+ 937us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:56.819810 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.821145 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.824023 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.825487 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.825690 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:56.825875 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:56.826501 (+ 626us) server_negotiation.cc:299] Negotiation successful
0928 14:53:56.826699 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13469,"thread_start_us":13304,"threads_started":1}
I20260928 14:53:56.827363 31311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:56.803475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.0.0.1:41070)
0928 14:53:56.817924 (+ 14449us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:56.817943 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:56.818084 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:56.818906 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:56.818914 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:56.818946 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:56.819402 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:56.819423 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.821312 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.821318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:56.823818 (+ 2500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:56.823831 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.825994 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.826000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:56.826222 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:56.826293 (+ 71us) client_negotiation.cc:769] Sending connection context
0928 14:53:56.826441 (+ 148us) client_negotiation.cc:240] Negotiation successful
0928 14:53:56.826906 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14316,"mutex_wait_us":148,"thread_start_us":10751,"threads_started":1}
I20260928 14:53:56.831357 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:56.805594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53280 (local address 127.14.241.195:42573)
0928 14:53:56.820849 (+ 15255us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:56.822681 (+ 1832us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:56.822761 (+ 80us) server_negotiation.cc:407] Connection header received
0928 14:53:56.823020 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:56.823029 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:56.823317 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:56.823497 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:56.824617 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.826130 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.828919 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.830311 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.830496 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:56.830673 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:56.830867 (+ 194us) server_negotiation.cc:299] Negotiation successful
0928 14:53:56.831024 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15066,"thread_start_us":14920,"threads_started":1}
I20260928 14:53:56.831409 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:56.804926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:42573 (local address 127.0.0.1:53280)
0928 14:53:56.822643 (+ 17717us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:56.822664 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:56.822821 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:56.823812 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:56.823816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:56.823892 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:56.824455 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:56.824467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.826277 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.826282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:56.828714 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:56.828725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.830496 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.830504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:56.830671 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:56.830707 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 14:53:56.830807 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 14:53:56.830974 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17544,"mutex_wait_us":145,"thread_start_us":17433,"threads_started":1}
I20260928 14:53:56.831905 31313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:56.803180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:45409 (local address 127.0.0.1:39132)
0928 14:53:56.814678 (+ 11498us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:56.814723 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:56.814946 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:56.823694 (+ 8748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:56.823699 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:56.823726 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:56.824266 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:56.824304 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.826104 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.826110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:56.828635 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:56.828657 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.830797 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.830804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:56.830988 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:56.831020 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 14:53:56.831105 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:53:56.831237 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11210,"mutex_wait_us":161,"thread_start_us":14449,"threads_started":1}
I20260928 14:53:56.832001 31312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:56.803380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39132 (local address 127.14.241.194:45409)
0928 14:53:56.823204 (+ 19824us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:56.823213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:56.823239 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:53:56.823319 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:56.823324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:56.823502 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:56.823682 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:56.824518 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.825971 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.828848 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:56.830279 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:56.830468 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:56.830639 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:56.831521 (+ 882us) server_negotiation.cc:299] Negotiation successful
0928 14:53:56.831693 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19666,"thread_start_us":19551,"threads_started":1}
I20260928 14:53:56.831916 30967 tablet_service.cc:1467] Processing CreateTablet for tablet 9e149977d9ef4ede80889ac7df35b1e1 (DEFAULT_TABLE table=test-table [id=00620aed0eec4c2a92bfdb2ba3897d0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:56.833635 30967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e149977d9ef4ede80889ac7df35b1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:56.835623 31234 tablet_service.cc:1467] Processing CreateTablet for tablet 9e149977d9ef4ede80889ac7df35b1e1 (DEFAULT_TABLE table=test-table [id=00620aed0eec4c2a92bfdb2ba3897d0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:56.835615 31101 tablet_service.cc:1467] Processing CreateTablet for tablet 9e149977d9ef4ede80889ac7df35b1e1 (DEFAULT_TABLE table=test-table [id=00620aed0eec4c2a92bfdb2ba3897d0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:53:56.837064 31234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e149977d9ef4ede80889ac7df35b1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:56.837064 31101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e149977d9ef4ede80889ac7df35b1e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:53:56.845220 31317 tablet_bootstrap.cc:492] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: Bootstrap starting.
I20260928 14:53:56.848997 31317 tablet_bootstrap.cc:654] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:56.850092 31317 log.cc:824] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:56.852792 31320 tablet_bootstrap.cc:492] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: Bootstrap starting.
I20260928 14:53:56.853029 31317 tablet_bootstrap.cc:492] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: No bootstrap required, opened a new log
I20260928 14:53:56.853091 31319 tablet_bootstrap.cc:492] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: Bootstrap starting.
I20260928 14:53:56.853327 31317 ts_tablet_manager.cc:1397] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 14:53:56.856529 31320 tablet_bootstrap.cc:654] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:56.856745 31319 tablet_bootstrap.cc:654] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: Neither blocks nor log segments found. Creating new log.
I20260928 14:53:56.857578 31320 log.cc:824] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:56.857949 31319 log.cc:824] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:53:56.860600 31319 tablet_bootstrap.cc:492] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: No bootstrap required, opened a new log
I20260928 14:53:56.860674 31320 tablet_bootstrap.cc:492] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: No bootstrap required, opened a new log
I20260928 14:53:56.860890 31319 ts_tablet_manager.cc:1397] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 14:53:56.860993 31320 ts_tablet_manager.cc:1397] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260928 14:53:56.863611 31317 raft_consensus.cc:348] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } }
I20260928 14:53:56.864316 31317 raft_consensus.cc:374] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:56.864449 31317 raft_consensus.cc:729] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e111d3231244499188d4f1c7157f72, State: Initialized, Role: FOLLOWER
I20260928 14:53:56.865132 31317 consensus_queue.cc:260] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } }
I20260928 14:53:56.868367 31317 ts_tablet_manager.cc:1428] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260928 14:53:56.869632 31319 raft_consensus.cc:348] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } }
I20260928 14:53:56.870136 31319 raft_consensus.cc:374] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:56.870244 31319 raft_consensus.cc:729] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f8b64d817e04c659f289970992c6fb2, State: Initialized, Role: FOLLOWER
I20260928 14:53:56.870361 31320 raft_consensus.cc:348] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } }
I20260928 14:53:56.870819 31320 raft_consensus.cc:374] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:53:56.870929 31320 raft_consensus.cc:729] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 333776926c6f4b71af88e695049f7d77, State: Initialized, Role: FOLLOWER
I20260928 14:53:56.870952 31319 consensus_queue.cc:260] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } }
I20260928 14:53:56.871488 31320 consensus_queue.cc:260] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } }
I20260928 14:53:56.872144 31166 heartbeater.cc:503] Master 127.14.241.254:41753 was elected leader, sending a full tablet report...
I20260928 14:53:56.872957 31319 ts_tablet_manager.cc:1428] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260928 14:53:56.873852 31299 heartbeater.cc:503] Master 127.14.241.254:41753 was elected leader, sending a full tablet report...
I20260928 14:53:56.874552 31320 ts_tablet_manager.cc:1428] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
W20260928 14:53:56.953038 31033 tablet.cc:2369] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:53:56.971992 31167 tablet.cc:2369] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:53:56.978255 31300 tablet.cc:2369] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:53:57.044682 31324 raft_consensus.cc:482] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:53:57.045022 31324 raft_consensus.cc:504] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } }
I20260928 14:53:57.046797 31324 leader_election.cc:288] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18e111d3231244499188d4f1c7157f72 (127.14.241.193:38353), 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573)
I20260928 14:53:57.053751 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:57.047327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:46855 (local address 127.14.241.195:42573)
0928 14:53:57.047529 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:57.047536 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:57.048035 (+ 499us) server_negotiation.cc:407] Connection header received
0928 14:53:57.048269 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:57.048302 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:57.048387 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:57.048456 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:57.049164 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.049979 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.051929 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.052774 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.052934 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:57.053030 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:57.053380 (+ 350us) server_negotiation.cc:299] Negotiation successful
0928 14:53:57.053511 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 14:53:57.053756 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:57.047180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:39501 (local address 127.14.241.193:38353)
0928 14:53:57.047418 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:57.047425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:57.048139 (+ 714us) server_negotiation.cc:407] Connection header received
0928 14:53:57.048328 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:57.048334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:57.048406 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:57.048486 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:53:57.049190 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.049930 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.051926 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.052767 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.052934 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:57.053011 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:57.053352 (+ 341us) server_negotiation.cc:299] Negotiation successful
0928 14:53:57.053489 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 14:53:57.054262 31327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:57.047203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.14.241.194:39501)
0928 14:53:57.048039 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:57.048053 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:57.048168 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:57.048520 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:57.048525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:57.048548 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:57.049041 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:57.049051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.050051 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.050056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:57.051766 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:57.051779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.053015 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.053021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:57.053180 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:57.053223 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 14:53:57.053299 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:53:57.053478 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":100,"thread_start_us":511,"threads_started":1}
I20260928 14:53:57.054296 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:57.047086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:42573 (local address 127.14.241.194:46855)
0928 14:53:57.047920 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0928 14:53:57.047935 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:53:57.048093 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:53:57.048497 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:53:57.048501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:53:57.048523 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:53:57.049016 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:57.049030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.050107 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.050111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:57.051765 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:53:57.051778 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.053049 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.053056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:53:57.053223 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:57.053246 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 14:53:57.053336 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 14:53:57.053466 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":58,"thread_start_us":495,"threads_started":1}
I20260928 14:53:57.055235 31254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e149977d9ef4ede80889ac7df35b1e1" candidate_uuid: "9f8b64d817e04c659f289970992c6fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "333776926c6f4b71af88e695049f7d77" is_pre_election: true
I20260928 14:53:57.055224 30987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e149977d9ef4ede80889ac7df35b1e1" candidate_uuid: "9f8b64d817e04c659f289970992c6fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e111d3231244499188d4f1c7157f72" is_pre_election: true
I20260928 14:53:57.055704 30987 raft_consensus.cc:2454] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f8b64d817e04c659f289970992c6fb2 in term 0.
I20260928 14:53:57.055704 31254 raft_consensus.cc:2454] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f8b64d817e04c659f289970992c6fb2 in term 0.
I20260928 14:53:57.056413 31055 leader_election.cc:302] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e111d3231244499188d4f1c7157f72, 9f8b64d817e04c659f289970992c6fb2; no voters:
I20260928 14:53:57.056797 31324 raft_consensus.cc:2781] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:53:57.056946 31324 raft_consensus.cc:482] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:53:57.057045 31324 raft_consensus.cc:3037] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:57.059163 31324 raft_consensus.cc:504] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } }
I20260928 14:53:57.059803 31324 leader_election.cc:288] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [CANDIDATE]: Term 1 election: Requested vote from peers 18e111d3231244499188d4f1c7157f72 (127.14.241.193:38353), 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573)
I20260928 14:53:57.060274 30987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e149977d9ef4ede80889ac7df35b1e1" candidate_uuid: "9f8b64d817e04c659f289970992c6fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e111d3231244499188d4f1c7157f72"
I20260928 14:53:57.060315 31254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e149977d9ef4ede80889ac7df35b1e1" candidate_uuid: "9f8b64d817e04c659f289970992c6fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "333776926c6f4b71af88e695049f7d77"
I20260928 14:53:57.060531 31254 raft_consensus.cc:3037] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:57.060530 30987 raft_consensus.cc:3037] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:53:57.063086 30987 raft_consensus.cc:2454] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f8b64d817e04c659f289970992c6fb2 in term 1.
I20260928 14:53:57.063086 31254 raft_consensus.cc:2454] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f8b64d817e04c659f289970992c6fb2 in term 1.
I20260928 14:53:57.063503 31055 leader_election.cc:302] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e111d3231244499188d4f1c7157f72, 9f8b64d817e04c659f289970992c6fb2; no voters:
I20260928 14:53:57.063819 31324 raft_consensus.cc:2781] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:53:57.064251 31324 raft_consensus.cc:686] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 1 LEADER]: Becoming Leader. State: Replica: 9f8b64d817e04c659f289970992c6fb2, State: Running, Role: LEADER
I20260928 14:53:57.064805 31324 consensus_queue.cc:237] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } }
I20260928 14:53:57.069728 30843 catalog_manager.cc:5495] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194). New cstate: current_term: 1 leader_uuid: "9f8b64d817e04c659f289970992c6fb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } health_report { overall_health: UNKNOWN } } }
I20260928 14:53:57.131609 31312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:57.126082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39148 (local address 127.14.241.194:45409)
0928 14:53:57.126283 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:57.126289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:57.126348 (+ 59us) server_negotiation.cc:407] Connection header received
0928 14:53:57.126569 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:57.126573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:57.126652 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:57.126829 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:53:57.127196 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.127944 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.129490 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.129891 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.130003 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:57.130492 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:53:57.131012 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:53:57.131149 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:57.131249 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 14:53:57.131338 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 14:53:57.139906 30987 raft_consensus.cc:1261] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Refusing update from remote peer 9f8b64d817e04c659f289970992c6fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:53:57.139911 31254 raft_consensus.cc:1261] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77 [term 1 FOLLOWER]: Refusing update from remote peer 9f8b64d817e04c659f289970992c6fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:53:57.140898 31328 consensus_queue.cc:1035] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:53:57.141222 31324 consensus_queue.cc:1035] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:53:57.168097 30843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:53:57.168419 30843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:53:57.171783 30843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Sending DeleteTablet for 3 replicas of tablet 9e149977d9ef4ede80889ac7df35b1e1
I20260928 14:53:57.172950 30967 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e149977d9ef4ede80889ac7df35b1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:57 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:41070
I20260928 14:53:57.172978 31234 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e149977d9ef4ede80889ac7df35b1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:57 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:53280
I20260928 14:53:57.173007 31101 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e149977d9ef4ede80889ac7df35b1e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:53:57 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:39132
I20260928 14:53:57.174002 31340 tablet_replica.cc:330] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: stopping tablet replica
I20260928 14:53:57.174178 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 30904
I20260928 14:53:57.174476 31340 raft_consensus.cc:2229] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:53:57.175105 31340 raft_consensus.cc:2258] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:53:57.176841 31341 tablet_replica.cc:330] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: stopping tablet replica
I20260928 14:53:57.177354 31341 raft_consensus.cc:2229] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:53:57.177754 31340 ts_tablet_manager.cc:1905] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:53:57.177917 31341 raft_consensus.cc:2258] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:53:57.180154 31341 ts_tablet_manager.cc:1905] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:53:57.185544 31340 ts_tablet_manager.cc:1918] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:53:57.185823 31340 log.cc:1196] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/9e149977d9ef4ede80889ac7df35b1e1
I20260928 14:53:57.186558 31340 ts_tablet_manager.cc:1939] T 9e149977d9ef4ede80889ac7df35b1e1 P 18e111d3231244499188d4f1c7157f72: Deleting consensus metadata
Sep 28 14:53:57 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:53:57.186928 31341 ts_tablet_manager.cc:1918] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:53:57.187129 31341 log.cc:1196] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/9e149977d9ef4ede80889ac7df35b1e1
I20260928 14:53:57.187615 31341 ts_tablet_manager.cc:1939] T 9e149977d9ef4ede80889ac7df35b1e1 P 333776926c6f4b71af88e695049f7d77: Deleting consensus metadata
I20260928 14:53:57.189031 30828 catalog_manager.cc:4841] TS 18e111d3231244499188d4f1c7157f72 (127.14.241.193:38353): tablet 9e149977d9ef4ede80889ac7df35b1e1 (table test-table [id=00620aed0eec4c2a92bfdb2ba3897d0c]) successfully deleted
I20260928 14:53:57.189492 30829 catalog_manager.cc:4841] TS 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573): tablet 9e149977d9ef4ede80889ac7df35b1e1 (table test-table [id=00620aed0eec4c2a92bfdb2ba3897d0c]) successfully deleted
I20260928 14:53:57.189795 31342 tablet_replica.cc:330] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: stopping tablet replica
I20260928 14:53:57.190275 31342 raft_consensus.cc:2229] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:53:57.191011 31342 raft_consensus.cc:2258] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:53:57.193151 31342 ts_tablet_manager.cc:1905] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:53:57.196172 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:57.175482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41076 (local address 127.14.241.193:38353)
0928 14:53:57.175654 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:57.175660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:57.176299 (+ 639us) server_negotiation.cc:407] Connection header received
0928 14:53:57.176475 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:57.176481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:57.176564 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:57.176652 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:57.178665 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.179811 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.181877 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.182528 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.182637 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:57.188179 (+ 5542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:57.188196 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:57.188215 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:57.188265 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:57.192020 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:57.192941 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:57.192951 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:57.192954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:57.193115 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:57.193576 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:57.193581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:57.193584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:57.194036 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:57.194319 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:57.195580 (+ 1261us) server_negotiation.cc:299] Negotiation successful
0928 14:53:57.195814 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:53:57.196897 31027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:57.200485 31342 ts_tablet_manager.cc:1918] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:53:57.200651 31342 log.cc:1196] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/9e149977d9ef4ede80889ac7df35b1e1
I20260928 14:53:57.201093 31342 ts_tablet_manager.cc:1939] T 9e149977d9ef4ede80889ac7df35b1e1 P 9f8b64d817e04c659f289970992c6fb2: Deleting consensus metadata
I20260928 14:53:57.202795 30830 catalog_manager.cc:4841] TS 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409): tablet 9e149977d9ef4ede80889ac7df35b1e1 (table test-table [id=00620aed0eec4c2a92bfdb2ba3897d0c]) successfully deleted
I20260928 14:53:57.333034 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 30904
I20260928 14:53:57.347918 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 31038
Sep 28 14:53:57 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:53:57.364547 31312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:57.348702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39150 (local address 127.14.241.194:45409)
0928 14:53:57.348892 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:57.348898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:57.348985 (+ 87us) server_negotiation.cc:407] Connection header received
0928 14:53:57.349167 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:57.349173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:57.349245 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:57.349346 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:57.350702 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.351534 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.353086 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.353573 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.353691 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:57.358199 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:57.358213 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:57.358230 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:57.358271 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:57.361499 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:57.362448 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:57.362456 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:57.362459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:57.362610 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:57.363015 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:57.363018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:57.363020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:57.363413 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:57.363673 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:57.363974 (+ 301us) server_negotiation.cc:299] Negotiation successful
0928 14:53:57.364143 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 14:53:57.365504 31161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:57.478623 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 31038
I20260928 14:53:57.494395 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 31171
Sep 28 14:53:57 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:53:57.509191 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:53:57.495084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53294 (local address 127.14.241.195:42573)
0928 14:53:57.495258 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 14:53:57.495265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:53:57.495352 (+ 87us) server_negotiation.cc:407] Connection header received
0928 14:53:57.495536 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:53:57.495541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:53:57.495614 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:53:57.495698 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:53:57.497077 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.497795 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.499195 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:53:57.499579 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:53:57.499687 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:53:57.503852 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:53:57.503869 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:53:57.503885 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:53:57.503925 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:53:57.506430 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:57.507120 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:57.507128 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:57.507130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:57.507276 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:53:57.507664 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:53:57.507669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:53:57.507672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:53:57.508034 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:53:57.508346 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0928 14:53:57.508634 (+ 288us) server_negotiation.cc:299] Negotiation successful
0928 14:53:57.508856 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:53:57.509810 31294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:57.597790 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 31171
I20260928 14:53:57.611805 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 30811
I20260928 14:53:57.612944 30873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:53:57.724712 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 30811
I20260928 14:54:17.742630 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41753
--webserver_interface=127.14.241.254
--webserver_port=33077
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:41753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:54:18.066260 31348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:54:18.066553 31348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:54:18.066686 31348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:54:18.073988 31348 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 14:54:18.074095 31348 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 14:54:18.074133 31348 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 14:54:18.074198 31348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:54:18.074270 31348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:54:18.074358 31348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:54:18.074409 31348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:54:18.074482 31348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:54:18.084633 31348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:41753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=33077
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:54:18.085951 31348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:54:18.087350 31348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:54:18.110378 31354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:18.110414 31357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:18.110330 31355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:54:18.110548 31348 server_base.cc:1034] running on GCE node
I20260928 14:54:18.111891 31348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:54:18.113623 31348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:54:18.114858 31348 hybrid_clock.cc:648] HybridClock initialized: now 1790607258114790 us; error 40 us; skew 500 ppm
I20260928 14:54:18.115238 31348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:54:18 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607258, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:54:18.123494 31348 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:54:18.125363 31348 webserver.cc:460] Webserver started at http://127.14.241.254:33077/ using document root <none> and password file <none>
I20260928 14:54:18.126154 31348 fs_manager.cc:359] Metadata directory not provided
I20260928 14:54:18.126279 31348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:54:18.132453 31348 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260928 14:54:18.135340 31364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:18.136768 31348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:54:18.136962 31348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "13ed2a70f062417b8e67e47350f71f64"
format_stamp: "Formatted at 2026-09-28 14:53:55 on dist-test-slave-vk40"
server_key: "be493fa344bc6a2a5e0bc8eab8c7ea12"
server_key_iv: "1a1787c168a92d1b3aac0a8a512f9743"
server_key_version: "encryptionkey@0"
I20260928 14:54:18.138075 31348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:54:18.161614 31348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:54:18.166036 31348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:54:18.166330 31348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:54:18.184896 31348 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:41753
I20260928 14:54:18.184954 31415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:41753 every 8 connection(s)
I20260928 14:54:18.186481 31348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:54:18.193801 31416 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:54:18.195420 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 31348
I20260928 14:54:18.196466 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:38353
--local_ip_for_outbound_sockets=127.14.241.193
--tserver_master_addrs=127.14.241.254:41753
--webserver_port=40827
--webserver_interface=127.14.241.193
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:54:18.198354 31416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Bootstrap starting.
I20260928 14:54:18.220252 31416 log.cc:824] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Log is configured to *not* fsync() on all Append() calls
I20260928 14:54:18.232741 31416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:54:18.233290 31416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Bootstrap complete.
I20260928 14:54:18.242784 31416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:54:18.243654 31416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ed2a70f062417b8e67e47350f71f64, State: Initialized, Role: FOLLOWER
I20260928 14:54:18.244311 31416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:54:18.244560 31416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:54:18.244673 31416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:54:18.244849 31416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:54:18.248827 31416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:54:18.249367 31416 leader_election.cc:302] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13ed2a70f062417b8e67e47350f71f64; no voters:
I20260928 14:54:18.250020 31416 leader_election.cc:288] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:54:18.250159 31420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:54:18.251322 31420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 2 LEADER]: Becoming Leader. State: Replica: 13ed2a70f062417b8e67e47350f71f64, State: Running, Role: LEADER
I20260928 14:54:18.251977 31420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:54:18.252419 31416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:54:18.254503 31421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "13ed2a70f062417b8e67e47350f71f64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } } }
I20260928 14:54:18.254981 31421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: This master's current role is: LEADER
I20260928 14:54:18.254875 31422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13ed2a70f062417b8e67e47350f71f64. Latest consensus state: current_term: 2 leader_uuid: "13ed2a70f062417b8e67e47350f71f64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } } }
I20260928 14:54:18.255196 31422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: This master's current role is: LEADER
I20260928 14:54:18.257136 31425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:54:18.266153 31425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:54:18.267704 31425 catalog_manager.cc:1259] Loaded cluster ID: 157fb1c8ed0842be842a650bd9510eb4
I20260928 14:54:18.267850 31425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:54:18.271981 31425 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:54:18.273643 31425 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Loaded TSK: 0
I20260928 14:54:18.304795 31437 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 14:54:18.311280 31425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Generated new TSK 1
I20260928 14:54:18.311777 31425 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:54:18.550603 31418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:54:18.550900 31418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:54:18.551012 31418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:54:18.558569 31418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:54:18.558681 31418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:54:18.558822 31418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:54:18.569085 31418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:38353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=40827
--tserver_master_addrs=127.14.241.254:41753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:54:18.570379 31418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:54:18.571717 31418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:54:18.585916 31443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:18.585937 31444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:18.586671 31446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:54:18.587474 31418 server_base.cc:1034] running on GCE node
I20260928 14:54:18.588126 31418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:54:18.589165 31418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:54:18.590389 31418 hybrid_clock.cc:648] HybridClock initialized: now 1790607258590335 us; error 48 us; skew 500 ppm
I20260928 14:54:18.590692 31418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:54:18 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607258, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:54:18.594033 31418 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:54:18.595600 31418 webserver.cc:460] Webserver started at http://127.14.241.193:40827/ using document root <none> and password file <none>
I20260928 14:54:18.596311 31418 fs_manager.cc:359] Metadata directory not provided
I20260928 14:54:18.596448 31418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:54:18.601751 31418 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260928 14:54:18.604562 31453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:18.605970 31418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 14:54:18.606117 31418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "18e111d3231244499188d4f1c7157f72"
format_stamp: "Formatted at 2026-09-28 14:53:55 on dist-test-slave-vk40"
server_key: "f22c61094b62edea6af65a85c90701bb"
server_key_iv: "debce66641de54b55a84cf549b43a363"
server_key_version: "encryptionkey@0"
I20260928 14:54:18.607111 31418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:54:18.622814 31418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:54:18.626953 31418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:54:18.627202 31418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:54:18.628141 31418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:54:18.629458 31418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:54:18.629554 31418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:18.629635 31418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:54:18.629709 31418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:18.659905 31418 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:38353
I20260928 14:54:18.659977 31565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:38353 every 8 connection(s)
I20260928 14:54:18.661386 31418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:54:18.663779 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 31418
I20260928 14:54:18.664592 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:45409
--local_ip_for_outbound_sockets=127.14.241.194
--tserver_master_addrs=127.14.241.254:41753
--webserver_port=45853
--webserver_interface=127.14.241.194
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:54:18 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607258, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:54:18.687130 31569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:18.664785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:59585)
0928 14:54:18.665719 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:18.665772 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 14:54:18.667543 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:54:18.669146 (+ 1603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:54:18.669170 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:54:18.669953 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:54:18.670552 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:18.670571 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:18.672219 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:18.672224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:18.674120 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:18.674137 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:18.675288 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:18.675295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:18.675555 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:18.676303 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:54:18.676333 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:54:18.678915 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:54:18.682637 (+ 3722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:54:18.682647 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:54:18.682663 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:54:18.683122 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:54:18.683590 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:54:18.683595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:54:18.683598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:54:18.683793 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:54:18.684590 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:54:18.684598 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:54:18.684945 (+ 347us) client_negotiation.cc:769] Sending connection context
0928 14:54:18.685183 (+ 238us) client_negotiation.cc:240] Negotiation successful
0928 14:54:18.685776 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":428,"threads_started":1}
I20260928 14:54:18.687130 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:18.664468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:59585 (local address 127.14.241.254:41753)
0928 14:54:18.665094 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:18.665109 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:18.665917 (+ 808us) server_negotiation.cc:407] Connection header received
0928 14:54:18.668129 (+ 2212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:18.668144 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:18.668588 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:18.668961 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:18.670770 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:18.672035 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:18.674316 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:18.675032 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:18.675179 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:18.679240 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:18.679249 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:18.679270 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:18.679313 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:18.682451 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:18.683278 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:18.683286 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:18.683292 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:18.683471 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:18.683912 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:18.683915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:18.683918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:18.684457 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:18.684652 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:18.685440 (+ 788us) server_negotiation.cc:299] Negotiation successful
0928 14:54:18.685847 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":259,"threads_started":1}
I20260928 14:54:18.690073 31566 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41753
I20260928 14:54:18.690377 31566 heartbeater.cc:464] Registering TS with master...
I20260928 14:54:18.691110 31566 heartbeater.cc:511] Master 127.14.241.254:41753 requested a full tablet report, sending...
I20260928 14:54:18.693265 31381 ts_manager.cc:194] Registered new tserver with Master: 18e111d3231244499188d4f1c7157f72 (127.14.241.193:38353)
I20260928 14:54:18.694801 31381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:59585
W20260928 14:54:18.993168 31570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:54:18.993479 31570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:54:18.993603 31570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:54:19.000823 31570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:54:19.000953 31570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:54:19.001111 31570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:54:19.010874 31570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:45409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=45853
--tserver_master_addrs=127.14.241.254:41753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:54:19.012193 31570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:54:19.013518 31570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:54:19.026422 31575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:19.026432 31576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:19.026356 31578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:54:19.034660 31570 server_base.cc:1034] running on GCE node
I20260928 14:54:19.035346 31570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:54:19.036342 31570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:54:19.037570 31570 hybrid_clock.cc:648] HybridClock initialized: now 1790607259037504 us; error 54 us; skew 500 ppm
I20260928 14:54:19.037910 31570 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:54:19 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607259, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:54:19.041190 31570 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:54:19.042798 31570 webserver.cc:460] Webserver started at http://127.14.241.194:45853/ using document root <none> and password file <none>
I20260928 14:54:19.043529 31570 fs_manager.cc:359] Metadata directory not provided
I20260928 14:54:19.043640 31570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:54:19.049204 31570 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 14:54:19.051980 31585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:19.053617 31570 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:54:19.053786 31570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "9f8b64d817e04c659f289970992c6fb2"
format_stamp: "Formatted at 2026-09-28 14:53:56 on dist-test-slave-vk40"
server_key: "0493eaa971846d4020e859ad5a4680a6"
server_key_iv: "474a876e287f8d344864205f07891b25"
server_key_version: "encryptionkey@0"
I20260928 14:54:19.054816 31570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:54:19.075968 31570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:54:19.080364 31570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:54:19.080626 31570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:54:19.081707 31570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:54:19.083104 31570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:54:19.083200 31570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:19.083303 31570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:54:19.083364 31570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:19.115250 31570 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:45409
I20260928 14:54:19.115342 31697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:45409 every 8 connection(s)
I20260928 14:54:19.117065 31570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:54:19.119740 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 31570
I20260928 14:54:19.120530 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:42573
--local_ip_for_outbound_sockets=127.14.241.195
--tserver_master_addrs=127.14.241.254:41753
--webserver_port=45027
--webserver_interface=127.14.241.195
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:54:19 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607259, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:54:19.138083 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.119499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:59391 (local address 127.14.241.254:41753)
0928 14:54:19.119675 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:19.119682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:19.121165 (+ 1483us) server_negotiation.cc:407] Connection header received
0928 14:54:19.123085 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:19.123091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:19.123176 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:19.123268 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:19.125048 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.125751 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.127363 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.127793 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.127916 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.132006 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:19.132012 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:19.132015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:19.132051 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:19.134523 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:19.135477 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:19.135482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:19.135484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:19.135619 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:19.136099 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:19.136103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:19.136105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:19.136506 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:19.136660 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:19.137613 (+ 953us) server_negotiation.cc:299] Negotiation successful
0928 14:54:19.137833 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:54:19.139595 31700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.120191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:59391)
0928 14:54:19.120995 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:19.121034 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:54:19.122754 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:54:19.123549 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:54:19.123559 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:54:19.124197 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:54:19.124856 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.124876 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.125917 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.125921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.127212 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.127226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.128033 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.128041 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.128365 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.129048 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:54:19.129082 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:54:19.131745 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:54:19.134691 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:54:19.134700 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:54:19.134720 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:54:19.135311 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:54:19.135725 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:54:19.135729 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:54:19.135733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:54:19.135979 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:54:19.136652 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:54:19.136660 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:54:19.137139 (+ 479us) client_negotiation.cc:769] Sending connection context
0928 14:54:19.137474 (+ 335us) client_negotiation.cc:240] Negotiation successful
0928 14:54:19.138062 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":292,"threads_started":1}
I20260928 14:54:19.141176 31698 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41753
I20260928 14:54:19.141494 31698 heartbeater.cc:464] Registering TS with master...
I20260928 14:54:19.142294 31698 heartbeater.cc:511] Master 127.14.241.254:41753 requested a full tablet report, sending...
I20260928 14:54:19.143695 31381 ts_manager.cc:194] Registered new tserver with Master: 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409)
I20260928 14:54:19.144560 31381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:59391
W20260928 14:54:19.453161 31701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:54:19.453635 31701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:54:19.453830 31701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:54:19.461282 31701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:54:19.461393 31701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:54:19.461539 31701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:54:19.471521 31701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:42573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=45027
--tserver_master_addrs=127.14.241.254:41753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:54:19.472808 31701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:54:19.474109 31701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:54:19.486711 31706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:19.486752 31709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:19.486778 31707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:54:19.487352 31701 server_base.cc:1034] running on GCE node
I20260928 14:54:19.487967 31701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:54:19.488914 31701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:54:19.490118 31701 hybrid_clock.cc:648] HybridClock initialized: now 1790607259490053 us; error 51 us; skew 500 ppm
I20260928 14:54:19.490437 31701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:54:19 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607259, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:54:19.493559 31701 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:54:19.495009 31701 webserver.cc:460] Webserver started at http://127.14.241.195:45027/ using document root <none> and password file <none>
I20260928 14:54:19.495674 31701 fs_manager.cc:359] Metadata directory not provided
I20260928 14:54:19.495806 31701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:54:19.501150 31701 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:54:19.503787 31716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:19.505190 31701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 14:54:19.505347 31701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "333776926c6f4b71af88e695049f7d77"
format_stamp: "Formatted at 2026-09-28 14:53:56 on dist-test-slave-vk40"
server_key: "d23479784a0ca640ff16cc6caac562da"
server_key_iv: "3ddd191ea3b67c3e64c7ec9933fe3a0c"
server_key_version: "encryptionkey@0"
I20260928 14:54:19.506330 31701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:54:19.519093 31701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:54:19.523062 31701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:54:19.523307 31701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:54:19.524230 31701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:54:19.525583 31701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:54:19.525686 31701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:19.525787 31701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:54:19.525862 31701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:19.555472 31701 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:42573
I20260928 14:54:19.555557 31828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:42573 every 8 connection(s)
I20260928 14:54:19.556985 31701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:54:19.566084 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 31701
Sep 28 14:54:19 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607259, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:54:19.578333 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.559569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:54205 (local address 127.14.241.254:41753)
0928 14:54:19.559744 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:19.559749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:19.560885 (+ 1136us) server_negotiation.cc:407] Connection header received
0928 14:54:19.562864 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:19.562870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:19.562954 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:19.563044 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:19.564723 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.565535 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.567350 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.567923 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.568001 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.572010 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:19.572015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:19.572018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:19.572052 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:19.574921 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:19.575799 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:19.575804 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:19.575807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:19.575967 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:19.576417 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:19.576421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:19.576423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:19.576773 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:19.576928 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:19.577716 (+ 788us) server_negotiation.cc:299] Negotiation successful
0928 14:54:19.577969 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:54:19.579689 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.560009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:54205)
0928 14:54:19.560705 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:19.560755 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:54:19.562576 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:54:19.563301 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:54:19.563311 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:54:19.563859 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:54:19.564549 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.564570 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.565676 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.565681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.567173 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.567189 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.568378 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.568384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.568589 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.569260 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:54:19.569283 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:54:19.571772 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:54:19.575075 (+ 3303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:54:19.575086 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:54:19.575107 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:54:19.575702 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:54:19.576070 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:54:19.576073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:54:19.576076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:54:19.576258 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:54:19.576911 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:54:19.576918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:54:19.577256 (+ 338us) client_negotiation.cc:769] Sending connection context
0928 14:54:19.577578 (+ 322us) client_negotiation.cc:240] Negotiation successful
0928 14:54:19.578136 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":283,"threads_started":1}
I20260928 14:54:19.581362 31829 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41753
I20260928 14:54:19.581439 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.567006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57404 (local address 127.14.241.254:41753)
0928 14:54:19.567840 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:19.567845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:19.567859 (+ 14us) server_negotiation.cc:407] Connection header received
0928 14:54:19.567909 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:19.567912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:19.567980 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:19.568068 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:19.569763 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.570705 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.572455 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.573058 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.573190 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.575842 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:19.575848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:19.575853 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:19.575885 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:19.578541 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:19.579329 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:19.579332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:19.579334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:19.579460 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:19.579988 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:19.579991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:19.579993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:19.580256 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:19.580532 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:19.580867 (+ 335us) server_negotiation.cc:299] Negotiation successful
0928 14:54:19.581084 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":575,"threads_started":1}
I20260928 14:54:19.581677 31829 heartbeater.cc:464] Registering TS with master...
I20260928 14:54:19.582525 31829 heartbeater.cc:511] Master 127.14.241.254:41753 requested a full tablet report, sending...
I20260928 14:54:19.584110 31381 ts_manager.cc:194] Registered new tserver with Master: 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573)
I20260928 14:54:19.584995 31381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:54205
I20260928 14:54:19.597193 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:54:19.603353 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.598052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57418 (local address 127.14.241.254:41753)
0928 14:54:19.598237 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:19.598242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:19.598648 (+ 406us) server_negotiation.cc:407] Connection header received
0928 14:54:19.598835 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:19.598839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:19.598982 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:19.599126 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:54:19.599486 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.600191 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.601639 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.602074 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.602162 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.602575 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:54:19.602800 (+ 225us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0928 14:54:19.603102 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57418: token expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260928 14:54:19.603463 31833 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57418: token expired
W20260928 14:54:19.603914 31304 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.14.241.254:41753)
I20260928 14:54:19.616211 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.604536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.14.241.254:41753)
0928 14:54:19.604690 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:19.604700 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:19.604717 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:54:19.604762 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:19.604765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:19.604823 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:19.604897 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:19.606239 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.606997 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.608342 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.608739 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.608829 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.611423 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:19.611428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:19.611432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:19.611468 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:19.613881 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:19.614500 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:19.614504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:19.614506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:19.614631 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:19.614972 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:19.614976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:19.614978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:19.615338 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:19.615537 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:19.615773 (+ 236us) server_negotiation.cc:299] Negotiation successful
0928 14:54:19.615946 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:54:19.619812 31381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:54:19.627081 31381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:54:19.666258 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.639067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.0.0.1:56106)
0928 14:54:19.655957 (+ 16890us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:19.655974 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:54:19.656090 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:54:19.657360 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:54:19.657364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:54:19.657390 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:54:19.657811 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.657831 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.659867 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.659873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.662524 (+ 2651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.662541 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.665190 (+ 2649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.665197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.665436 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.665494 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 14:54:19.665608 (+ 114us) client_negotiation.cc:240] Negotiation successful
0928 14:54:19.665817 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16768,"mutex_wait_us":6,"thread_start_us":16615,"threads_started":1}
I20260928 14:54:19.666816 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.638664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:42573 (local address 127.0.0.1:34582)
0928 14:54:19.653985 (+ 15321us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:19.654016 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 14:54:19.654169 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:54:19.656414 (+ 2245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:54:19.656423 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:54:19.656462 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:54:19.656923 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.656936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.659588 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.659594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.662675 (+ 3081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.662692 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.665898 (+ 3206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.665904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.666107 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.666135 (+ 28us) client_negotiation.cc:769] Sending connection context
0928 14:54:19.666197 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 14:54:19.666313 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15164,"mutex_wait_us":651,"thread_start_us":12669,"threads_started":1}
I20260928 14:54:19.666898 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.638810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44866 (local address 127.14.241.194:45409)
0928 14:54:19.657322 (+ 18512us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:19.657331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:19.657369 (+ 38us) server_negotiation.cc:407] Connection header received
0928 14:54:19.657468 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:19.657475 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:19.657651 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:19.657814 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:54:19.658319 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.660908 (+ 2589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.663328 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.665202 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.665383 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.665572 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:19.666353 (+ 781us) server_negotiation.cc:299] Negotiation successful
0928 14:54:19.666529 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18264,"thread_start_us":18248,"threads_started":1}
I20260928 14:54:19.666992 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.638498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:45409 (local address 127.0.0.1:44866)
0928 14:54:19.651796 (+ 13298us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:19.651840 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:54:19.652045 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:54:19.657830 (+ 5785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:54:19.657834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:54:19.657857 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:54:19.658173 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.658182 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.661066 (+ 2884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.661072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.663149 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.663162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.665754 (+ 2592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.665761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.665945 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.665981 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 14:54:19.666063 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:54:19.666198 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12978,"mutex_wait_us":1,"thread_start_us":15767,"threads_started":1}
I20260928 14:54:19.667899 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.639789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56106 (local address 127.14.241.193:38353)
0928 14:54:19.654954 (+ 15165us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:19.654964 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:19.656298 (+ 1334us) server_negotiation.cc:407] Connection header received
0928 14:54:19.656433 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:19.656440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:19.656685 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:19.657181 (+ 496us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:54:19.658067 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.659716 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.662759 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.664853 (+ 2094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.667118 (+ 2265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.667302 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:19.667368 (+ 66us) server_negotiation.cc:299] Negotiation successful
0928 14:54:19.667529 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14972,"thread_start_us":14843,"threads_started":1}
I20260928 14:54:19.668638 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.639286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.14.241.195:42573)
0928 14:54:19.655616 (+ 16330us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:19.655626 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:19.655667 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:54:19.655791 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:19.655801 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:19.656102 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:19.657156 (+ 1054us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:54:19.657445 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.659311 (+ 1866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.662874 (+ 3563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.665410 (+ 2536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.665581 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.665753 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:19.668112 (+ 2359us) server_negotiation.cc:299] Negotiation successful
0928 14:54:19.668324 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16078,"thread_start_us":15950,"threads_started":1}
I20260928 14:54:19.670845 31633 tablet_service.cc:1467] Processing CreateTablet for tablet fcabd78d67f846f7bda286a08cc71c3e (DEFAULT_TABLE table=test-table [id=d14577484e224c91bd514129d4323664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:54:19.671211 31501 tablet_service.cc:1467] Processing CreateTablet for tablet fcabd78d67f846f7bda286a08cc71c3e (DEFAULT_TABLE table=test-table [id=d14577484e224c91bd514129d4323664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:54:19.672341 31633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcabd78d67f846f7bda286a08cc71c3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:54:19.672448 31501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcabd78d67f846f7bda286a08cc71c3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:54:19.672317 31764 tablet_service.cc:1467] Processing CreateTablet for tablet fcabd78d67f846f7bda286a08cc71c3e (DEFAULT_TABLE table=test-table [id=d14577484e224c91bd514129d4323664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:54:19.673877 31764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcabd78d67f846f7bda286a08cc71c3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:54:19.685462 31845 tablet_bootstrap.cc:492] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Bootstrap starting.
I20260928 14:54:19.688010 31846 tablet_bootstrap.cc:492] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: Bootstrap starting.
I20260928 14:54:19.689734 31847 tablet_bootstrap.cc:492] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: Bootstrap starting.
I20260928 14:54:19.691417 31845 tablet_bootstrap.cc:654] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Neither blocks nor log segments found. Creating new log.
I20260928 14:54:19.691706 31846 tablet_bootstrap.cc:654] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: Neither blocks nor log segments found. Creating new log.
I20260928 14:54:19.692833 31846 log.cc:824] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:54:19.692833 31845 log.cc:824] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Log is configured to *not* fsync() on all Append() calls
I20260928 14:54:19.693001 31847 tablet_bootstrap.cc:654] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: Neither blocks nor log segments found. Creating new log.
I20260928 14:54:19.694185 31847 log.cc:824] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: Log is configured to *not* fsync() on all Append() calls
I20260928 14:54:19.696312 31846 tablet_bootstrap.cc:492] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: No bootstrap required, opened a new log
I20260928 14:54:19.696307 31845 tablet_bootstrap.cc:492] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: No bootstrap required, opened a new log
I20260928 14:54:19.696599 31845 ts_tablet_manager.cc:1397] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260928 14:54:19.696599 31846 ts_tablet_manager.cc:1397] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260928 14:54:19.696877 31847 tablet_bootstrap.cc:492] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: No bootstrap required, opened a new log
I20260928 14:54:19.697113 31847 ts_tablet_manager.cc:1397] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260928 14:54:19.697897 31566 heartbeater.cc:503] Master 127.14.241.254:41753 was elected leader, sending a full tablet report...
I20260928 14:54:19.705199 31847 raft_consensus.cc:348] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:19.705591 31847 raft_consensus.cc:374] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:54:19.705696 31847 raft_consensus.cc:729] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 333776926c6f4b71af88e695049f7d77, State: Initialized, Role: FOLLOWER
I20260928 14:54:19.706293 31847 consensus_queue.cc:260] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:19.707726 31829 heartbeater.cc:503] Master 127.14.241.254:41753 was elected leader, sending a full tablet report...
I20260928 14:54:19.707795 31845 raft_consensus.cc:348] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:19.708242 31845 raft_consensus.cc:374] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:54:19.708276 31847 ts_tablet_manager.cc:1428] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260928 14:54:19.708371 31845 raft_consensus.cc:729] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e111d3231244499188d4f1c7157f72, State: Initialized, Role: FOLLOWER
I20260928 14:54:19.708990 31845 consensus_queue.cc:260] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:19.710780 31846 raft_consensus.cc:348] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:19.711221 31845 ts_tablet_manager.cc:1428] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260928 14:54:19.711288 31846 raft_consensus.cc:374] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:54:19.711398 31846 raft_consensus.cc:729] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f8b64d817e04c659f289970992c6fb2, State: Initialized, Role: FOLLOWER
I20260928 14:54:19.711987 31846 consensus_queue.cc:260] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:19.714015 31698 heartbeater.cc:503] Master 127.14.241.254:41753 was elected leader, sending a full tablet report...
I20260928 14:54:19.714720 31846 ts_tablet_manager.cc:1428] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: Time spent starting tablet: real 0.018s user 0.010s sys 0.003s
W20260928 14:54:19.808549 31830 tablet.cc:2369] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:54:19.868976 31699 tablet.cc:2369] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:54:19.893146 31851 raft_consensus.cc:482] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:54:19.893435 31851 raft_consensus.cc:504] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:19.895107 31851 leader_election.cc:288] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409), 18e111d3231244499188d4f1c7157f72 (127.14.241.193:38353)
I20260928 14:54:19.901340 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.895578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:55235 (local address 127.14.241.194:45409)
0928 14:54:19.895806 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:19.895811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:19.895928 (+ 117us) server_negotiation.cc:407] Connection header received
0928 14:54:19.896139 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:19.896144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:19.896213 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:19.896318 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:54:19.896722 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.897648 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.899646 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.900464 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.900599 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.900685 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:19.900956 (+ 271us) server_negotiation.cc:299] Negotiation successful
0928 14:54:19.901087 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:54:19.901340 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.895612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:45409 (local address 127.14.241.195:55235)
0928 14:54:19.895848 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:19.895863 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:54:19.895978 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:54:19.896357 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:54:19.896362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:54:19.896391 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:54:19.896576 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.896585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.897776 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.897779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.899480 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.899495 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.900612 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.900618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.900808 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.900833 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 14:54:19.900913 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 14:54:19.901045 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260928 14:54:19.902092 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.895613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:41395 (local address 127.14.241.193:38353)
0928 14:54:19.895804 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:19.895809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:19.897002 (+ 1193us) server_negotiation.cc:407] Connection header received
0928 14:54:19.897110 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:19.897115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:19.897189 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:19.897280 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:54:19.897761 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.898588 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.900495 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.901245 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.901399 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.901471 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:19.901786 (+ 315us) server_negotiation.cc:299] Negotiation successful
0928 14:54:19.901922 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260928 14:54:19.902187 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.896162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.14.241.195:41395)
0928 14:54:19.896872 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:19.896894 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 14:54:19.897007 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:54:19.897297 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:54:19.897300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:54:19.897315 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:54:19.897600 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.897608 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.898736 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.898742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.900344 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:19.900357 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.901439 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.901445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:19.901626 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.901655 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 14:54:19.901738 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:54:19.901856 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":388,"threads_started":1}
I20260928 14:54:19.902432 31653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcabd78d67f846f7bda286a08cc71c3e" candidate_uuid: "333776926c6f4b71af88e695049f7d77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f8b64d817e04c659f289970992c6fb2" is_pre_election: true
I20260928 14:54:19.902930 31653 raft_consensus.cc:2454] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 333776926c6f4b71af88e695049f7d77 in term 0.
I20260928 14:54:19.903128 31521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcabd78d67f846f7bda286a08cc71c3e" candidate_uuid: "333776926c6f4b71af88e695049f7d77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e111d3231244499188d4f1c7157f72" is_pre_election: true
I20260928 14:54:19.903637 31720 leader_election.cc:302] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 333776926c6f4b71af88e695049f7d77, 9f8b64d817e04c659f289970992c6fb2; no voters:
I20260928 14:54:19.903645 31521 raft_consensus.cc:2454] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 333776926c6f4b71af88e695049f7d77 in term 0.
I20260928 14:54:19.903985 31851 raft_consensus.cc:2781] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:54:19.904139 31851 raft_consensus.cc:482] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:54:19.904253 31851 raft_consensus.cc:3037] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:54:19.908435 31851 raft_consensus.cc:504] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:19.909142 31851 leader_election.cc:288] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [CANDIDATE]: Term 1 election: Requested vote from peers 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409), 18e111d3231244499188d4f1c7157f72 (127.14.241.193:38353)
I20260928 14:54:19.909732 31653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcabd78d67f846f7bda286a08cc71c3e" candidate_uuid: "333776926c6f4b71af88e695049f7d77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f8b64d817e04c659f289970992c6fb2"
I20260928 14:54:19.909780 31521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcabd78d67f846f7bda286a08cc71c3e" candidate_uuid: "333776926c6f4b71af88e695049f7d77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e111d3231244499188d4f1c7157f72"
I20260928 14:54:19.909945 31653 raft_consensus.cc:3037] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:54:19.909994 31521 raft_consensus.cc:3037] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:54:19.912432 31653 raft_consensus.cc:2454] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 333776926c6f4b71af88e695049f7d77 in term 1.
I20260928 14:54:19.912484 31521 raft_consensus.cc:2454] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 333776926c6f4b71af88e695049f7d77 in term 1.
I20260928 14:54:19.912921 31720 leader_election.cc:302] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 18e111d3231244499188d4f1c7157f72, 333776926c6f4b71af88e695049f7d77, 9f8b64d817e04c659f289970992c6fb2; no voters:
I20260928 14:54:19.913266 31851 raft_consensus.cc:2781] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 1 FOLLOWER]: Leader election won for term 1
W20260928 14:54:19.913617 31567 tablet.cc:2369] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:54:19.913749 31851 raft_consensus.cc:686] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 1 LEADER]: Becoming Leader. State: Replica: 333776926c6f4b71af88e695049f7d77, State: Running, Role: LEADER
I20260928 14:54:19.914458 31851 consensus_queue.cc:237] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:19.918918 31380 catalog_manager.cc:5495] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 333776926c6f4b71af88e695049f7d77 (127.14.241.195). New cstate: current_term: 1 leader_uuid: "333776926c6f4b71af88e695049f7d77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } health_report { overall_health: UNKNOWN } } }
I20260928 14:54:19.968564 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:19.963000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34586 (local address 127.14.241.195:42573)
0928 14:54:19.963189 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:19.963194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:19.963213 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:54:19.963404 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:19.963408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:19.963488 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:19.963616 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:54:19.964005 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.964824 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.966377 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:19.966793 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:19.966904 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:19.967436 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:54:19.967946 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:54:19.968092 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:19.968193 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 14:54:19.968298 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:54:19.976759 31653 raft_consensus.cc:1261] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2 [term 1 FOLLOWER]: Refusing update from remote peer 333776926c6f4b71af88e695049f7d77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:54:19.976764 31521 raft_consensus.cc:1261] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Refusing update from remote peer 333776926c6f4b71af88e695049f7d77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:54:19.977702 31855 consensus_queue.cc:1035] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:54:19.978026 31851 consensus_queue.cc:1035] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:54:19.989105 31858 mvcc.cc:205] Tried to move back new op lower bound from 7334327336854196224 to 7334327336613277696. Current Snapshot: MvccSnapshot[applied={T|T < 7334327336854196224}]
I20260928 14:54:19.991146 31859 mvcc.cc:205] Tried to move back new op lower bound from 7334327336854196224 to 7334327336613277696. Current Snapshot: MvccSnapshot[applied={T|T < 7334327336854196224}]
I20260928 14:54:20.004329 31380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:54:20.004618 31380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:54:20.008236 31380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Sending DeleteTablet for 3 replicas of tablet fcabd78d67f846f7bda286a08cc71c3e
I20260928 14:54:20.009457 31764 tablet_service.cc:1514] Processing DeleteTablet for tablet fcabd78d67f846f7bda286a08cc71c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:54:20 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:34582
I20260928 14:54:20.009459 31633 tablet_service.cc:1514] Processing DeleteTablet for tablet fcabd78d67f846f7bda286a08cc71c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:54:20 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:44866
I20260928 14:54:20.009799 31501 tablet_service.cc:1514] Processing DeleteTablet for tablet fcabd78d67f846f7bda286a08cc71c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:54:20 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:56106
I20260928 14:54:20.010422 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 31418
I20260928 14:54:20.026832 31869 tablet_replica.cc:330] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: stopping tablet replica
I20260928 14:54:20.027392 31869 raft_consensus.cc:2229] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:54:20.028306 31869 raft_consensus.cc:2258] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:54:20.030745 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:20.011591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.14.241.193:38353)
0928 14:54:20.011743 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:20.011748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:20.011766 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:54:20.011927 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:20.011931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:20.012009 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:20.012100 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:20.013891 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:20.015172 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:20.017270 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:20.018019 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:20.018138 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:20.022085 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:20.022105 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:20.022127 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:20.022187 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:20.026337 (+ 4150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:20.027414 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:20.027423 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:20.027427 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:20.027611 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:20.028099 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:20.028104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:20.028106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:20.028651 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:20.028954 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:20.030137 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0928 14:54:20.030375 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:54:20.030985 31869 ts_tablet_manager.cc:1905] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:54:20.031447 31561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:54:20.037832 31870 tablet_replica.cc:330] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: stopping tablet replica
I20260928 14:54:20.038287 31870 raft_consensus.cc:2229] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:54:20.038828 31870 raft_consensus.cc:2258] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:54:20.038934 31869 ts_tablet_manager.cc:1918] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:54:20.039098 31869 log.cc:1196] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/fcabd78d67f846f7bda286a08cc71c3e
I20260928 14:54:20.040753 31869 ts_tablet_manager.cc:1939] T fcabd78d67f846f7bda286a08cc71c3e P 333776926c6f4b71af88e695049f7d77: Deleting consensus metadata
I20260928 14:54:20.041489 31870 ts_tablet_manager.cc:1905] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:54:20.042986 31367 catalog_manager.cc:4841] TS 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573): tablet fcabd78d67f846f7bda286a08cc71c3e (table test-table [id=d14577484e224c91bd514129d4323664]) successfully deleted
I20260928 14:54:20.046969 31870 ts_tablet_manager.cc:1918] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:54:20.047122 31870 log.cc:1196] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/fcabd78d67f846f7bda286a08cc71c3e
I20260928 14:54:20.047567 31870 ts_tablet_manager.cc:1939] T fcabd78d67f846f7bda286a08cc71c3e P 9f8b64d817e04c659f289970992c6fb2: Deleting consensus metadata
I20260928 14:54:20.049098 31368 catalog_manager.cc:4841] TS 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409): tablet fcabd78d67f846f7bda286a08cc71c3e (table test-table [id=d14577484e224c91bd514129d4323664]) successfully deleted
I20260928 14:54:20.154681 31871 tablet_replica.cc:330] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: stopping tablet replica
I20260928 14:54:20.154938 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 31418
W20260928 14:54:20.171051 31366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.241.193:38353 (error 108)
I20260928 14:54:20.171697 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 31570
I20260928 14:54:20.173236 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:20.172497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.0.0.1:56128)
0928 14:54:20.172916 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:20.173042 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":207,"threads_started":1}
W20260928 14:54:20.173555 31366 catalog_manager.cc:4584] TS 18e111d3231244499188d4f1c7157f72 (127.14.241.193:38353): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fcabd78d67f846f7bda286a08cc71c3e: Network error: Client connection negotiation failed: client connection to 127.14.241.193:38353: connect: Connection refused (error 111)
I20260928 14:54:20.193352 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:20.172501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44878 (local address 127.14.241.194:45409)
0928 14:54:20.173005 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:20.173012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:20.173032 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:54:20.173097 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:20.173102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:20.173183 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:20.173301 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:20.175015 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:20.175743 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:20.177832 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:20.178231 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:20.178323 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:20.186219 (+ 7896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:20.186239 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:20.186259 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:20.186310 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:20.190064 (+ 3754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:20.190968 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:20.190979 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:20.190982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:20.191137 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:20.191536 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:20.191541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:20.191545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:20.192050 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:20.192377 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:20.192737 (+ 360us) server_negotiation.cc:299] Negotiation successful
0928 14:54:20.192977 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379}
I20260928 14:54:20.194036 31693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:54:20.227283 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:20.226692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.0.0.1:56136)
0928 14:54:20.226961 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:20.227092 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260928 14:54:20.228677 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:20.228207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.0.0.1:56138)
0928 14:54:20.228406 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:20.228504 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260928 14:54:20.287658 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 31570
I20260928 14:54:20.290491 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:20.289978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.0.0.1:56142)
0928 14:54:20.290207 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:20.290315 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260928 14:54:20.291658 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:20.291274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.0.0.1:56148)
0928 14:54:20.291429 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:20.291510 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260928 14:54:20.302723 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 31701
I20260928 14:54:20.316896 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:20.303457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34596 (local address 127.14.241.195:42573)
0928 14:54:20.303640 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:20.303647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:20.303778 (+ 131us) server_negotiation.cc:407] Connection header received
0928 14:54:20.303953 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:20.303959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:20.304039 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:20.304134 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:20.305482 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:20.306282 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:20.307589 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:20.308034 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:20.308150 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:20.310904 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:20.310921 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:20.310939 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:20.310984 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:20.314118 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:20.314806 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:20.314813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:20.314816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:20.314982 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:20.315346 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:20.315349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:20.315352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:20.315718 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:20.316023 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:20.316306 (+ 283us) server_negotiation.cc:299] Negotiation successful
0928 14:54:20.316513 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 14:54:20.317528 31824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:54:20.372385 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:20.371796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.0.0.1:56150)
0928 14:54:20.372050 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:20.372165 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260928 14:54:20.373559 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:20.373064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.0.0.1:56166)
0928 14:54:20.373280 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:20.373373 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260928 14:54:20.442052 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 31701
I20260928 14:54:20.458333 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 31348
I20260928 14:54:20.459570 31411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:54:20.574221 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 31348
I20260928 14:54:20.574299 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:20.573760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:38353 (local address 127.0.0.1:56168)
0928 14:54:20.573959 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:20.574092 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260928 14:54:40.592931 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41753
--webserver_interface=127.14.241.254
--webserver_port=33077
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:41753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:54:40.922161 31878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:54:40.922472 31878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:54:40.922562 31878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:54:40.930025 31878 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 14:54:40.930138 31878 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 14:54:40.930176 31878 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 14:54:40.930230 31878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:54:40.930291 31878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:54:40.930359 31878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:54:40.930403 31878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:54:40.930457 31878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:54:40.940619 31878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:41753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=33077
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:54:40.941864 31878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:54:40.943184 31878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:54:40.954663 31883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:40.954694 31884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:40.954790 31886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:54:40.955339 31878 server_base.cc:1034] running on GCE node
I20260928 14:54:40.956063 31878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:54:40.957464 31878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:54:40.958676 31878 hybrid_clock.cc:648] HybridClock initialized: now 1790607280958596 us; error 47 us; skew 500 ppm
I20260928 14:54:40.958994 31878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:54:40 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607280, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:54:40.966130 31878 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:54:40.967682 31878 webserver.cc:460] Webserver started at http://127.14.241.254:33077/ using document root <none> and password file <none>
I20260928 14:54:40.968433 31878 fs_manager.cc:359] Metadata directory not provided
I20260928 14:54:40.968529 31878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:54:40.973933 31878 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:54:40.976748 31893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:40.978163 31878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 14:54:40.978343 31878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "13ed2a70f062417b8e67e47350f71f64"
format_stamp: "Formatted at 2026-09-28 14:53:55 on dist-test-slave-vk40"
server_key: "be493fa344bc6a2a5e0bc8eab8c7ea12"
server_key_iv: "1a1787c168a92d1b3aac0a8a512f9743"
server_key_version: "encryptionkey@0"
I20260928 14:54:40.979296 31878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:54:40.998442 31878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:54:41.002761 31878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:54:41.003036 31878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:54:41.020948 31878 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:41753
I20260928 14:54:41.020993 31944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:41753 every 8 connection(s)
I20260928 14:54:41.022466 31878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:54:41.026278 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 31878
I20260928 14:54:41.027107 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:38353
--local_ip_for_outbound_sockets=127.14.241.193
--tserver_master_addrs=127.14.241.254:41753
--webserver_port=40827
--webserver_interface=127.14.241.193
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:54:41.029579 31945 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:54:41.033034 31945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Bootstrap starting.
I20260928 14:54:41.053432 31945 log.cc:824] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Log is configured to *not* fsync() on all Append() calls
I20260928 14:54:41.069437 31945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:54:41.069953 31945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Bootstrap complete.
I20260928 14:54:41.078775 31945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:54:41.079667 31945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ed2a70f062417b8e67e47350f71f64, State: Initialized, Role: FOLLOWER
I20260928 14:54:41.080241 31945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:54:41.080509 31945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:54:41.080618 31945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:54:41.080772 31945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 2 FOLLOWER]: Advancing to term 3
I20260928 14:54:41.084769 31945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:54:41.085228 31945 leader_election.cc:302] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13ed2a70f062417b8e67e47350f71f64; no voters:
I20260928 14:54:41.085832 31945 leader_election.cc:288] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 14:54:41.085995 31949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 3 FOLLOWER]: Leader election won for term 3
I20260928 14:54:41.087023 31949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [term 3 LEADER]: Becoming Leader. State: Replica: 13ed2a70f062417b8e67e47350f71f64, State: Running, Role: LEADER
I20260928 14:54:41.087567 31949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } }
I20260928 14:54:41.087895 31945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:54:41.090173 31950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "13ed2a70f062417b8e67e47350f71f64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } } }
I20260928 14:54:41.090550 31950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: This master's current role is: LEADER
I20260928 14:54:41.094345 31951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13ed2a70f062417b8e67e47350f71f64. Latest consensus state: current_term: 3 leader_uuid: "13ed2a70f062417b8e67e47350f71f64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed2a70f062417b8e67e47350f71f64" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41753 } } }
I20260928 14:54:41.094664 31951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64 [sys.catalog]: This master's current role is: LEADER
I20260928 14:54:41.096165 31955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:54:41.104611 31955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:54:41.106268 31955 catalog_manager.cc:1259] Loaded cluster ID: 157fb1c8ed0842be842a650bd9510eb4
I20260928 14:54:41.106632 31955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:54:41.111662 31955 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:54:41.114261 31955 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Loaded TSK: 0
I20260928 14:54:41.114588 31955 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 13ed2a70f062417b8e67e47350f71f64: Loaded TSK: 1
F20260928 14:54:41.122138 31955 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f19e7ce59ad google::LogMessage::Fail() at ??:0
@ 0x7f19e7ce888a google::LogMessage::SendToLog() at ??:0
@ 0x7f19e7ce54de google::LogMessage::Flush() at ??:0
@ 0x7f19e7ce6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f19eba497d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f19eba3873e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f19fd7141c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f19fd713783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f19fd79f2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f19fd79f0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f19fd7e95e9 std::function<>::operator()() at ??:0
@ 0x7f19fd717cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f19fd715c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f19fd79d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f19fd79d439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f19ea931345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f19ea93491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f19ea934799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f19ea90feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f19edbe96db start_thread at ??:0
@ 0x7f19e6b7f61f clone at ??:0
W20260928 14:54:41.431774 31947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:54:41.432078 31947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:54:41.432204 31947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:54:41.439683 31947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:54:41.439805 31947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:54:41.439951 31947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:54:41.450043 31947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:38353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=40827
--tserver_master_addrs=127.14.241.254:41753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:54:41.451361 31947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:54:41.452744 31947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:54:41.465902 31995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:41.465875 31993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:41.465832 31992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:54:41.466475 31947 server_base.cc:1034] running on GCE node
I20260928 14:54:41.467149 31947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:54:41.468158 31947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:54:41.469391 31947 hybrid_clock.cc:648] HybridClock initialized: now 1790607281469339 us; error 44 us; skew 500 ppm
I20260928 14:54:41.469732 31947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:54:41 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607281, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:54:41.472976 31947 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:54:41.474504 31947 webserver.cc:460] Webserver started at http://127.14.241.193:40827/ using document root <none> and password file <none>
I20260928 14:54:41.475175 31947 fs_manager.cc:359] Metadata directory not provided
I20260928 14:54:41.475286 31947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:54:41.480931 31947 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260928 14:54:41.483678 32002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:41.485088 31947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:54:41.485242 31947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "18e111d3231244499188d4f1c7157f72"
format_stamp: "Formatted at 2026-09-28 14:53:55 on dist-test-slave-vk40"
server_key: "f22c61094b62edea6af65a85c90701bb"
server_key_iv: "debce66641de54b55a84cf549b43a363"
server_key_version: "encryptionkey@0"
I20260928 14:54:41.486181 31947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:54:41.518671 31947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:54:41.522897 31947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:54:41.523168 31947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:54:41.524170 31947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:54:41.526022 32009 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260928 14:54:41.531801 31947 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 14:54:41.531949 31947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260928 14:54:41.532071 31947 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 14:54:41.535209 31947 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 14:54:41.535316 31947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260928 14:54:41.535626 32009 tablet_bootstrap.cc:492] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Bootstrap starting.
I20260928 14:54:41.561935 32009 log.cc:824] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Log is configured to *not* fsync() on all Append() calls
I20260928 14:54:41.571280 31947 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:38353
I20260928 14:54:41.571359 32116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:38353 every 8 connection(s)
I20260928 14:54:41.572588 31947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:54:41.574203 32009 tablet_bootstrap.cc:492] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:54:41.574681 32009 tablet_bootstrap.cc:492] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Bootstrap complete.
I20260928 14:54:41.575762 32009 ts_tablet_manager.cc:1397] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Time spent bootstrapping tablet: real 0.040s user 0.028s sys 0.008s
I20260928 14:54:41.577354 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:41.575234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:59357)
0928 14:54:41.576042 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:41.576394 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":348,"threads_started":1}
W20260928 14:54:41.577951 32117 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:41753 (0 consecutive failures): Network error: Failed to ping master at 127.14.241.254:41753: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
I20260928 14:54:41.578871 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:41.578534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:36253)
0928 14:54:41.578681 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:41.578754 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260928 14:54:41.579756 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 31947
I20260928 14:54:41.579860 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:41.579430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:49287)
0928 14:54:41.579595 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:41.579701 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260928 14:54:41.580124 32117 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:54:41.580525 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:45409
--local_ip_for_outbound_sockets=127.14.241.194
--tserver_master_addrs=127.14.241.254:41753
--webserver_port=45853
--webserver_interface=127.14.241.194
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:54:41.583253 32009 raft_consensus.cc:348] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:41.584156 32009 raft_consensus.cc:729] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e111d3231244499188d4f1c7157f72, State: Initialized, Role: FOLLOWER
I20260928 14:54:41.584818 32009 consensus_queue.cc:260] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:41.586351 32009 ts_tablet_manager.cc:1428] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72: Time spent starting tablet: real 0.010s user 0.009s sys 0.004s
I20260928 14:54:41.586596 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:41.586162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:39337)
0928 14:54:41.586343 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:41.586439 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260928 14:54:41.587756 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:41.587186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:46717)
0928 14:54:41.587361 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:41.587584 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260928 14:54:41.917778 32121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:54:41.918118 32121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:54:41.918254 32121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:54:41.925616 32121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:54:41.925725 32121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:54:41.925884 32121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:54:41.935711 32121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:45409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=45853
--tserver_master_addrs=127.14.241.254:41753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:54:41.937052 32121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:54:41.938354 32121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:54:41.951591 32127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:41.951764 32126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:41.951661 32129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:54:41.952562 32121 server_base.cc:1034] running on GCE node
I20260928 14:54:41.953202 32121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:54:41.954177 32121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:54:41.955410 32121 hybrid_clock.cc:648] HybridClock initialized: now 1790607281955354 us; error 41 us; skew 500 ppm
I20260928 14:54:41.955758 32121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:54:41 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607281, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:54:41.958930 32121 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:54:41.960430 32121 webserver.cc:460] Webserver started at http://127.14.241.194:45853/ using document root <none> and password file <none>
I20260928 14:54:41.961105 32121 fs_manager.cc:359] Metadata directory not provided
I20260928 14:54:41.961221 32121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:54:41.966781 32121 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 14:54:41.970981 32136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:41.972414 32121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:54:41.972570 32121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "9f8b64d817e04c659f289970992c6fb2"
format_stamp: "Formatted at 2026-09-28 14:53:56 on dist-test-slave-vk40"
server_key: "0493eaa971846d4020e859ad5a4680a6"
server_key_iv: "474a876e287f8d344864205f07891b25"
server_key_version: "encryptionkey@0"
I20260928 14:54:41.973546 32121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:54:41.992071 32121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:54:41.995993 32121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:54:41.996235 32121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:54:41.997257 32121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:54:41.998528 32121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:54:41.998622 32121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:41.998737 32121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:54:41.998803 32121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:42.028329 32121 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:45409
I20260928 14:54:42.028381 32248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:45409 every 8 connection(s)
I20260928 14:54:42.029820 32121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:54:42.034801 32251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:42.032810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:44883)
0928 14:54:42.033540 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:42.033804 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":276,"threads_started":1}
I20260928 14:54:42.034952 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 32121
W20260928 14:54:42.035560 32249 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:41753 (0 consecutive failures): Network error: Failed to ping master at 127.14.241.254:41753: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
I20260928 14:54:42.035725 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:42573
--local_ip_for_outbound_sockets=127.14.241.195
--tserver_master_addrs=127.14.241.254:41753
--webserver_port=45027
--webserver_interface=127.14.241.195
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:54:42.036598 32251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:42.036171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:45893)
0928 14:54:42.036375 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:42.036466 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260928 14:54:42.037462 32251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:42.037133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:48217)
0928 14:54:42.037261 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:42.037343 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260928 14:54:42.037714 32249 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:54:42.352427 32252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:54:42.352757 32252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:54:42.352893 32252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:54:42.360205 32252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:54:42.360364 32252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:54:42.360523 32252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:54:42.370637 32252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:42573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=45027
--tserver_master_addrs=127.14.241.254:41753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:54:42.371987 32252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:54:42.373323 32252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:54:42.386230 32260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:42.386277 32258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:42.386210 32257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:54:42.387056 32252 server_base.cc:1034] running on GCE node
I20260928 14:54:42.387650 32252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:54:42.388649 32252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:54:42.389858 32252 hybrid_clock.cc:648] HybridClock initialized: now 1790607282389796 us; error 46 us; skew 500 ppm
I20260928 14:54:42.390170 32252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:54:42 dist-test-slave-vk40 krb5kdc[30798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607282, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:54:42.393177 32252 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:54:42.394691 32252 webserver.cc:460] Webserver started at http://127.14.241.195:45027/ using document root <none> and password file <none>
I20260928 14:54:42.395370 32252 fs_manager.cc:359] Metadata directory not provided
I20260928 14:54:42.395483 32252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:54:42.400849 32252 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 14:54:42.403522 32267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:42.404911 32252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:54:42.405062 32252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "333776926c6f4b71af88e695049f7d77"
format_stamp: "Formatted at 2026-09-28 14:53:56 on dist-test-slave-vk40"
server_key: "d23479784a0ca640ff16cc6caac562da"
server_key_iv: "3ddd191ea3b67c3e64c7ec9933fe3a0c"
server_key_version: "encryptionkey@0"
I20260928 14:54:42.406024 32252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:54:42.423450 32252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:54:42.427747 32252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:54:42.428002 32252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:54:42.429031 32252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:54:42.430398 32252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:54:42.430486 32252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:42.430589 32252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:54:42.430657 32252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:42.460572 32252 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:42573
I20260928 14:54:42.460638 32379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:42573 every 8 connection(s)
I20260928 14:54:42.462036 32252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:54:42.466950 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:42.464984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:33471)
0928 14:54:42.465808 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:42.466052 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":358,"threads_started":1}
W20260928 14:54:42.467639 32380 heartbeater.cc:644] Failed to heartbeat to 127.14.241.254:41753 (0 consecutive failures): Network error: Failed to ping master at 127.14.241.254:41753: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
I20260928 14:54:42.467903 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 32252
I20260928 14:54:42.468609 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:42.468227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:58883)
0928 14:54:42.468417 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:42.468503 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260928 14:54:42.469388 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:42.469082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:44123)
0928 14:54:42.469211 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:42.469281 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260928 14:54:42.469617 32380 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:54:42.589458 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:42.588652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:35683)
0928 14:54:42.589134 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:42.589236 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":221,"threads_started":1}
I20260928 14:54:42.791481 32385 raft_consensus.cc:482] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:54:42.791759 32385 raft_consensus.cc:504] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:42.793690 32385 leader_election.cc:288] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409), 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573)
Sep 28 14:54:42 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607281, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
Sep 28 14:54:42 dist-test-slave-vk40 krb5kdc[30798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607281, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:54:42.815018 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:42.793952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:42573 (local address 127.14.241.193:57387)
0928 14:54:42.795312 (+ 1360us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:42.795337 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 14:54:42.795877 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:54:42.798420 (+ 2543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:54:42.798428 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:54:42.799070 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:54:42.799693 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:42.799713 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:42.801377 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:42.801381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:42.802664 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:42.802683 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:42.803634 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:42.803641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:42.803945 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:42.804717 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:54:42.804747 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:54:42.807244 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:54:42.811266 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:54:42.811275 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:54:42.811290 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:54:42.811724 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:54:42.812171 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:54:42.812175 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:54:42.812179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:54:42.812422 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:54:42.813351 (+ 929us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:54:42.813357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:54:42.813705 (+ 348us) client_negotiation.cc:769] Sending connection context
0928 14:54:42.814029 (+ 324us) client_negotiation.cc:240] Negotiation successful
0928 14:54:42.814543 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1209,"mutex_wait_us":7,"spinlock_wait_cycles":13568}
I20260928 14:54:42.815070 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:42.793971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:45409 (local address 127.14.241.193:56877)
0928 14:54:42.795306 (+ 1335us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:42.795337 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 14:54:42.795871 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:54:42.800721 (+ 4850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:54:42.800739 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:54:42.801219 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:54:42.801591 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:42.801606 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:42.803052 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:42.803056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:42.804081 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:54:42.804096 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:54:42.805187 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:54:42.805193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:54:42.805360 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:42.805945 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:54:42.805968 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:54:42.808147 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:54:42.811723 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:54:42.811728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:54:42.811732 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:54:42.812082 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:54:42.812535 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:54:42.812539 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:54:42.812541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:54:42.812658 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:54:42.813510 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:54:42.813514 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:54:42.813726 (+ 212us) client_negotiation.cc:769] Sending connection context
0928 14:54:42.814029 (+ 303us) client_negotiation.cc:240] Negotiation successful
0928 14:54:42.814671 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218,"mutex_wait_us":895,"spinlock_wait_cycles":64512,"thread_start_us":339,"threads_started":1}
I20260928 14:54:42.815471 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:42.794379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:57387 (local address 127.14.241.195:42573)
0928 14:54:42.795112 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:42.795131 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:42.795477 (+ 346us) server_negotiation.cc:407] Connection header received
0928 14:54:42.797601 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:42.797609 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:42.797999 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:42.798270 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:42.799894 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:42.801183 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:42.802817 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:42.803442 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:42.803611 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:42.807529 (+ 3918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:42.807540 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:42.807559 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:42.807611 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:42.811125 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:42.811857 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:42.811863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:42.811868 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:42.812053 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:42.812527 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:42.812532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:42.812536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:42.813205 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:42.813393 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:42.814255 (+ 862us) server_negotiation.cc:299] Negotiation successful
0928 14:54:42.814680 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":349,"threads_started":1}
I20260928 14:54:42.815063 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:42.794202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:56877 (local address 127.14.241.194:45409)
0928 14:54:42.795917 (+ 1715us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:42.795928 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:42.795960 (+ 32us) server_negotiation.cc:407] Connection header received
0928 14:54:42.800044 (+ 4084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:42.800054 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:42.800438 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:42.800738 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:42.801758 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:42.802903 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:42.804263 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:42.804988 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:42.805148 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:42.808478 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:42.808489 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:42.808508 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:42.808557 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:42.811586 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:42.812208 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:42.812214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:42.812219 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:42.812425 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:42.812767 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:42.812772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:42.812775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:42.813389 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:42.813534 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:42.814255 (+ 721us) server_negotiation.cc:299] Negotiation successful
0928 14:54:42.814681 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1558,"thread_start_us":1440,"threads_started":1}
W20260928 14:54:42.819177 32005 leader_election.cc:341] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573): Not found: Tablet not found: fcabd78d67f846f7bda286a08cc71c3e
W20260928 14:54:42.819502 32006 leader_election.cc:341] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409): Not found: Tablet not found: fcabd78d67f846f7bda286a08cc71c3e
I20260928 14:54:42.819643 32006 leader_election.cc:302] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e111d3231244499188d4f1c7157f72; no voters: 333776926c6f4b71af88e695049f7d77, 9f8b64d817e04c659f289970992c6fb2
I20260928 14:54:42.820044 32385 raft_consensus.cc:2726] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:54:43.039305 32389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:43.038380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:48281)
0928 14:54:43.038885 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:43.039061 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":249,"threads_started":1}
I20260928 14:54:43.470942 32390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:43.470207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:34143)
0928 14:54:43.470632 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:43.470745 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":195,"threads_started":1}
I20260928 14:54:43.591084 32391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:43.590395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:53747)
0928 14:54:43.590807 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:43.590911 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":206,"threads_started":1}
I20260928 14:54:44.040975 32392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:44.040216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:55405)
0928 14:54:44.040707 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:44.040797 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":266,"threads_started":1}
I20260928 14:54:44.472594 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:44.471824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:45155)
0928 14:54:44.472311 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:44.472414 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":229,"threads_started":1}
I20260928 14:54:44.592767 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:44.591966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:45251)
0928 14:54:44.592488 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:44.592572 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":214,"threads_started":1}
I20260928 14:54:44.750851 32395 raft_consensus.cc:482] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:54:44.751056 32395 raft_consensus.cc:504] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:44.751749 32395 leader_election.cc:288] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409), 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573)
W20260928 14:54:44.752799 32006 leader_election.cc:341] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409): Not found: Tablet not found: fcabd78d67f846f7bda286a08cc71c3e
W20260928 14:54:44.752991 32005 leader_election.cc:341] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573): Not found: Tablet not found: fcabd78d67f846f7bda286a08cc71c3e
I20260928 14:54:44.753196 32005 leader_election.cc:302] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e111d3231244499188d4f1c7157f72; no voters: 333776926c6f4b71af88e695049f7d77, 9f8b64d817e04c659f289970992c6fb2
I20260928 14:54:44.753494 32395 raft_consensus.cc:2726] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:54:45.042680 32396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:45.041872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:48235)
0928 14:54:45.042357 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:45.042474 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":218,"threads_started":1}
I20260928 14:54:45.474107 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:45.473410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:44669)
0928 14:54:45.473844 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:45.473944 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":221,"threads_started":1}
I20260928 14:54:45.594254 32398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:45.593576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:33155)
0928 14:54:45.594008 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:45.594093 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":212,"threads_started":1}
I20260928 14:54:46.044317 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:46.043581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:39459)
0928 14:54:46.044026 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:46.044106 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":223,"threads_started":1}
I20260928 14:54:46.475687 32400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:46.474981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:40147)
0928 14:54:46.475400 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:46.475499 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":198,"threads_started":1}
I20260928 14:54:46.596040 32401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:46.595342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:36805)
0928 14:54:46.595774 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:46.595861 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":217,"threads_started":1}
I20260928 14:54:46.977548 32402 raft_consensus.cc:482] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:54:46.977762 32402 raft_consensus.cc:504] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:46.978466 32402 leader_election.cc:288] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409), 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573)
W20260928 14:54:46.979485 32006 leader_election.cc:341] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409): Not found: Tablet not found: fcabd78d67f846f7bda286a08cc71c3e
W20260928 14:54:46.979643 32005 leader_election.cc:341] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573): Not found: Tablet not found: fcabd78d67f846f7bda286a08cc71c3e
I20260928 14:54:46.979761 32005 leader_election.cc:302] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e111d3231244499188d4f1c7157f72; no voters: 333776926c6f4b71af88e695049f7d77, 9f8b64d817e04c659f289970992c6fb2
I20260928 14:54:46.979980 32402 raft_consensus.cc:2726] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:54:47.045984 32403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:47.045261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:33083)
0928 14:54:47.045712 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:47.045798 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":231,"threads_started":1}
I20260928 14:54:47.477538 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:47.476635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:37641)
0928 14:54:47.477221 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:47.477315 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":271,"threads_started":1}
I20260928 14:54:47.597620 32405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:47.596899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:44423)
0928 14:54:47.597339 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:47.597454 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":238,"threads_started":1}
I20260928 14:54:48.047684 32406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:48.046931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:36837)
0928 14:54:48.047416 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:48.047509 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":253,"threads_started":1}
I20260928 14:54:48.479156 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:48.478391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:54705)
0928 14:54:48.478897 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:48.478981 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":286,"threads_started":1}
I20260928 14:54:48.599220 32408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:48.598465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:43223)
0928 14:54:48.598927 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:48.599017 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":237,"threads_started":1}
I20260928 14:54:49.049500 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:49.048681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:43545)
0928 14:54:49.049211 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:49.049313 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":248,"threads_started":1}
I20260928 14:54:49.480799 32410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:49.480013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:43537)
0928 14:54:49.480514 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:49.480596 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":220,"threads_started":1}
I20260928 14:54:49.600813 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:49.600110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:38377)
0928 14:54:49.600562 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:49.600646 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":242,"threads_started":1}
I20260928 14:54:50.051221 32412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:50.050462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:35403)
0928 14:54:50.050942 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:50.051032 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":226,"threads_started":1}
I20260928 14:54:50.482349 32413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:50.481631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:42003)
0928 14:54:50.482077 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:50.482165 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":240,"threads_started":1}
I20260928 14:54:50.602662 32414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:50.601836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:58285)
0928 14:54:50.602357 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:50.602469 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":240,"threads_started":1}
I20260928 14:54:50.650490 32415 raft_consensus.cc:482] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:54:50.650715 32415 raft_consensus.cc:504] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:50.651376 32415 leader_election.cc:288] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409), 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573)
W20260928 14:54:50.652308 32006 leader_election.cc:341] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409): Not found: Tablet not found: fcabd78d67f846f7bda286a08cc71c3e
W20260928 14:54:50.652470 32005 leader_election.cc:341] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573): Not found: Tablet not found: fcabd78d67f846f7bda286a08cc71c3e
I20260928 14:54:50.652582 32005 leader_election.cc:302] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e111d3231244499188d4f1c7157f72; no voters: 333776926c6f4b71af88e695049f7d77, 9f8b64d817e04c659f289970992c6fb2
I20260928 14:54:50.652812 32415 raft_consensus.cc:2726] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:54:51.052995 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:51.052217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:39435)
0928 14:54:51.052733 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:51.052838 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":254,"threads_started":1}
I20260928 14:54:51.484067 32417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:51.483260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:42149)
0928 14:54:51.483782 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:51.483894 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":235,"threads_started":1}
I20260928 14:54:51.604593 32418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:51.603809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:52627)
0928 14:54:51.604323 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:51.604423 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":238,"threads_started":1}
I20260928 14:54:52.054559 32419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:52.053866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:38665)
0928 14:54:52.054288 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:52.054370 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":221,"threads_started":1}
I20260928 14:54:52.485699 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:52.484989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:48125)
0928 14:54:52.485440 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:52.485529 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":218,"threads_started":1}
I20260928 14:54:52.606225 32421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:52.605488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:34235)
0928 14:54:52.605932 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:52.606026 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":211,"threads_started":1}
I20260928 14:54:53.056520 32422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:53.055534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:47767)
0928 14:54:53.056171 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:53.056334 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":325,"threads_started":1}
I20260928 14:54:53.487380 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:53.486597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:60201)
0928 14:54:53.487106 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:53.487204 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":249,"threads_started":1}
I20260928 14:54:53.607888 32424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:53.607179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:55123)
0928 14:54:53.607616 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:53.607702 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":223,"threads_started":1}
I20260928 14:54:54.058238 32425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:54.057526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:35603)
0928 14:54:54.057991 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:54.058069 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":263,"threads_started":1}
I20260928 14:54:54.489177 32426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:54.488451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:59613)
0928 14:54:54.488912 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:54.489005 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":240,"threads_started":1}
I20260928 14:54:54.609629 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:54.608844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:58619)
0928 14:54:54.609317 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:54.609407 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":225,"threads_started":1}
I20260928 14:54:55.059796 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:55.059073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:33957)
0928 14:54:55.059543 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:55.059635 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":237,"threads_started":1}
I20260928 14:54:55.490908 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:55.490086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:42793)
0928 14:54:55.490580 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:55.490679 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":259,"threads_started":1}
I20260928 14:54:55.611291 32430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:55.610569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:43133)
0928 14:54:55.611005 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:55.611102 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":213,"threads_started":1}
I20260928 14:54:56.061527 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:56.060749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:52487)
0928 14:54:56.061211 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:56.061310 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":218,"threads_started":1}
I20260928 14:54:56.492530 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:56.491768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:34895)
0928 14:54:56.492214 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:56.492330 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":228,"threads_started":1}
I20260928 14:54:56.529160 32433 raft_consensus.cc:482] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:54:56.529373 32433 raft_consensus.cc:504] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f8b64d817e04c659f289970992c6fb2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 45409 } } peers { permanent_uuid: "333776926c6f4b71af88e695049f7d77" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42573 } } peers { permanent_uuid: "18e111d3231244499188d4f1c7157f72" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 38353 } }
I20260928 14:54:56.530045 32433 leader_election.cc:288] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409), 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573)
W20260928 14:54:56.530962 32006 leader_election.cc:341] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f8b64d817e04c659f289970992c6fb2 (127.14.241.194:45409): Not found: Tablet not found: fcabd78d67f846f7bda286a08cc71c3e
W20260928 14:54:56.531116 32005 leader_election.cc:341] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 333776926c6f4b71af88e695049f7d77 (127.14.241.195:42573): Not found: Tablet not found: fcabd78d67f846f7bda286a08cc71c3e
I20260928 14:54:56.531267 32005 leader_election.cc:302] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18e111d3231244499188d4f1c7157f72; no voters: 333776926c6f4b71af88e695049f7d77, 9f8b64d817e04c659f289970992c6fb2
I20260928 14:54:56.531503 32433 raft_consensus.cc:2726] T fcabd78d67f846f7bda286a08cc71c3e P 18e111d3231244499188d4f1c7157f72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:54:56.613320 32434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:56.612493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.193:33807)
0928 14:54:56.612965 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:56.613107 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":216,"threads_started":1}
I20260928 14:54:57.063525 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:57.062719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.194:40055)
0928 14:54:57.063229 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:57.063363 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":272,"threads_started":1}
I20260928 14:54:57.478343 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 31947
I20260928 14:54:57.494372 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:57.493638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41753 (local address 127.14.241.195:45431)
0928 14:54:57.494089 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0928 14:54:57.494204 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.254:41753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":209,"threads_started":1}
I20260928 14:54:57.497184 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:57.479478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58748 (local address 127.14.241.193:38353)
0928 14:54:57.480015 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:57.480024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:57.480394 (+ 370us) server_negotiation.cc:407] Connection header received
0928 14:54:57.480597 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:57.480607 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:57.480881 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:57.481018 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:57.482351 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:57.483276 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:57.484561 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:57.485109 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:57.485208 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:57.491670 (+ 6462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:57.491684 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:57.491698 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:57.491738 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:57.494832 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:57.495466 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:57.495472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:57.495475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:57.495597 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:57.495865 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:57.495868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:57.495870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:57.496200 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:57.496489 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:57.496674 (+ 185us) server_negotiation.cc:299] Negotiation successful
0928 14:54:57.496870 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":224,"threads_started":1}
I20260928 14:54:57.498358 32112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:54:57.587635 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 31947
I20260928 14:54:57.602429 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 32121
I20260928 14:54:57.615263 32439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:57.603168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.14.241.194:45409)
0928 14:54:57.603593 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:57.603598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:57.603614 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:54:57.603672 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:57.603675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:57.603737 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:57.603824 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:57.605230 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:57.605975 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:57.607346 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:57.607774 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:57.607871 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:57.610303 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:57.610310 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:57.610314 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:57.610355 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:57.612946 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:57.613564 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:57.613567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:57.613570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:57.613691 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:57.613967 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:57.613970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:57.613971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:57.614217 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:57.614420 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:57.614732 (+ 312us) server_negotiation.cc:299] Negotiation successful
0928 14:54:57.614918 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":219,"threads_started":1}
I20260928 14:54:57.616003 32244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:54:57.699115 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 32121
I20260928 14:54:57.713688 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 32252
I20260928 14:54:57.726588 32441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:57.714456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44594 (local address 127.14.241.195:42573)
0928 14:54:57.714986 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:57.714990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:57.715007 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:54:57.715064 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:57.715068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:57.715126 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:57.715215 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:57.716518 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:57.717314 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:57.718449 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:57.718903 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:57.719001 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:57.721481 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:57.721486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:57.721489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:57.721523 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:57.724127 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:57.724783 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:57.724786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:57.724788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:57.724926 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:57.725301 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:57.725303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:57.725305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:57.725566 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:57.725791 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:57.726082 (+ 291us) server_negotiation.cc:299] Negotiation successful
0928 14:54:57.726252 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":257,"threads_started":1}
I20260928 14:54:57.727288 32375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:54:57.811442 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 32252
2026-09-28T14:54:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64428 ms)
[----------] 30 tests from SecurityITest (292946 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:54:57 dist-test-slave-vk40 krb5kdc[32445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:54:57 dist-test-slave-vk40 krb5kdc[32445](info): set up 2 sockets
Sep 28 14:54:57 dist-test-slave-vk40 krb5kdc[32445](info): commencing operation
krb5kdc: starting...
W20260928 14:54:58.835067 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:54:58 dist-test-slave-vk40 krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:54:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:54:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:54:59.075845 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:34579
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:46805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:34579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:54:59.404013 32458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:54:59.404361 32458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:54:59.404511 32458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:54:59.411885 32458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:54:59.411995 32458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:54:59.412047 32458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:54:59.412124 32458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:54:59.412181 32458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:54:59.422283 32458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:46805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:34579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:34579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:54:59.423612 32458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:54:59.425004 32458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:54:59.436842 32463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:54:59.436914 32466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:54:59.436957 32458 server_base.cc:1034] running on GCE node
W20260928 14:54:59.436893 32464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:54:59.438055 32458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:54:59.439456 32458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:54:59.440645 32458 hybrid_clock.cc:648] HybridClock initialized: now 1790607299440592 us; error 41 us; skew 500 ppm
I20260928 14:54:59.441004 32458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:54:59 dist-test-slave-vk40 krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607299, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:54:59.443962 32458 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:54:59.445456 32458 webserver.cc:460] Webserver started at http://127.14.241.254:33427/ using document root <none> and password file <none>
I20260928 14:54:59.446125 32458 fs_manager.cc:359] Metadata directory not provided
I20260928 14:54:59.446240 32458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:54:59.446527 32458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:54:59.449540 32458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "c002bbb4203140289dad6c55f13ee0c6"
format_stamp: "Formatted at 2026-09-28 14:54:59 on dist-test-slave-vk40"
server_key: "c7b080e0ab1d636f5b4d13fd4e579a36"
server_key_iv: "ec52759d8e714d79704d466379af93f9"
server_key_version: "encryptionkey@0"
I20260928 14:54:59.450296 32458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "c002bbb4203140289dad6c55f13ee0c6"
format_stamp: "Formatted at 2026-09-28 14:54:59 on dist-test-slave-vk40"
server_key: "c7b080e0ab1d636f5b4d13fd4e579a36"
server_key_iv: "ec52759d8e714d79704d466379af93f9"
server_key_version: "encryptionkey@0"
I20260928 14:54:59.455433 32458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:54:59.458920 32473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:54:59.460275 32458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:54:59.460455 32458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "c002bbb4203140289dad6c55f13ee0c6"
format_stamp: "Formatted at 2026-09-28 14:54:59 on dist-test-slave-vk40"
server_key: "c7b080e0ab1d636f5b4d13fd4e579a36"
server_key_iv: "ec52759d8e714d79704d466379af93f9"
server_key_version: "encryptionkey@0"
I20260928 14:54:59.460623 32458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:54:59.475458 32458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:54:59.479377 32458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:54:59.479610 32458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:54:59.496801 32458 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:34579
I20260928 14:54:59.496810 32524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:34579 every 8 connection(s)
I20260928 14:54:59.498112 32458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:54:59.498620 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 32458
I20260928 14:54:59.498819 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:54:59.499550 15303 external_mini_cluster.cc:1428] Setting key ed9aaaca81374945716739d7647db01c
I20260928 14:54:59.502369 32525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 14:54:59 dist-test-slave-vk40 krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:54:59.513096 32525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6: Bootstrap starting.
I20260928 14:54:59.516520 32525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6: Neither blocks nor log segments found. Creating new log.
I20260928 14:54:59.517565 32525 log.cc:824] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6: Log is configured to *not* fsync() on all Append() calls
I20260928 14:54:59.520423 32525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6: No bootstrap required, opened a new log
I20260928 14:54:59.521044 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:54:59.501454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33008 (local address 127.14.241.254:34579)
0928 14:54:59.502219 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0928 14:54:59.502231 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:54:59.502269 (+ 38us) server_negotiation.cc:407] Connection header received
0928 14:54:59.503913 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:54:59.503934 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:54:59.504363 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:54:59.504738 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:54:59.506166 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:59.507561 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:59.508953 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:54:59.509631 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:54:59.509824 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:54:59.513823 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:54:59.513832 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:54:59.513847 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:54:59.513890 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:54:59.517043 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:59.517739 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:59.517745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:59.517752 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:59.517926 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:54:59.518286 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:54:59.518289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:54:59.518291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:54:59.518839 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:54:59.519121 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0928 14:54:59.519569 (+ 448us) server_negotiation.cc:299] Negotiation successful
0928 14:54:59.519985 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":364,"threads_started":1}
I20260928 14:54:59.528980 32525 raft_consensus.cc:348] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c002bbb4203140289dad6c55f13ee0c6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34579 } }
I20260928 14:54:59.529346 32525 raft_consensus.cc:374] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:54:59.529438 32525 raft_consensus.cc:729] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c002bbb4203140289dad6c55f13ee0c6, State: Initialized, Role: FOLLOWER
I20260928 14:54:59.529985 32525 consensus_queue.cc:260] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c002bbb4203140289dad6c55f13ee0c6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34579 } }
I20260928 14:54:59.530191 32525 raft_consensus.cc:388] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:54:59.530294 32525 raft_consensus.cc:482] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:54:59.530429 32525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:54:59.532920 32525 raft_consensus.cc:504] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c002bbb4203140289dad6c55f13ee0c6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34579 } }
I20260928 14:54:59.533387 32525 leader_election.cc:302] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c002bbb4203140289dad6c55f13ee0c6; no voters:
I20260928 14:54:59.533936 32525 leader_election.cc:288] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:54:59.534055 32530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:54:59.534680 32530 raft_consensus.cc:686] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [term 1 LEADER]: Becoming Leader. State: Replica: c002bbb4203140289dad6c55f13ee0c6, State: Running, Role: LEADER
I20260928 14:54:59.535295 32530 consensus_queue.cc:237] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c002bbb4203140289dad6c55f13ee0c6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34579 } }
I20260928 14:54:59.535930 32525 sys_catalog.cc:564] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:54:59.538004 32532 sys_catalog.cc:455] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c002bbb4203140289dad6c55f13ee0c6. Latest consensus state: current_term: 1 leader_uuid: "c002bbb4203140289dad6c55f13ee0c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c002bbb4203140289dad6c55f13ee0c6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34579 } } }
I20260928 14:54:59.538419 32532 sys_catalog.cc:458] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [sys.catalog]: This master's current role is: LEADER
I20260928 14:54:59.538321 32531 sys_catalog.cc:455] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c002bbb4203140289dad6c55f13ee0c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c002bbb4203140289dad6c55f13ee0c6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34579 } } }
I20260928 14:54:59.538578 32531 sys_catalog.cc:458] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6 [sys.catalog]: This master's current role is: LEADER
I20260928 14:54:59.540344 32539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:54:59.545190 32539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:54:59.553468 32539 catalog_manager.cc:1347] Generated new cluster ID: 1ec588429da741638ce8a7a436e01bd0
I20260928 14:54:59.553623 32539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:54:59.569133 32539 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:54:59.570982 32539 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:54:59.579002 32539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6: Generated new TSK 0
I20260928 14:54:59.580058 32539 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:54:59.635641 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34579
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:46805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:54:59.970726 32551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:54:59.971027 32551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:54:59.971123 32551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:54:59.978474 32551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:54:59.978585 32551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:54:59.978698 32551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:54:59.988734 32551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:46805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:54:59.989991 32551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:54:59.991271 32551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:00.004617 32557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:00.004580 32559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:00.004643 32556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:00.005290 32551 server_base.cc:1034] running on GCE node
I20260928 14:55:00.005901 32551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:00.006815 32551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:00.008002 32551 hybrid_clock.cc:648] HybridClock initialized: now 1790607300007946 us; error 52 us; skew 500 ppm
I20260928 14:55:00.008342 32551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:00 dist-test-slave-vk40 krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607300, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:00.011485 32551 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:55:00.013063 32551 webserver.cc:460] Webserver started at http://127.14.241.193:36763/ using document root <none> and password file <none>
I20260928 14:55:00.013731 32551 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:00.013842 32551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:00.014104 32551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:00.017028 32551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "b7b9cba3a6fa41be999febd380d84440"
format_stamp: "Formatted at 2026-09-28 14:55:00 on dist-test-slave-vk40"
server_key: "2368c385be6b4dfe523a38fd22476f7f"
server_key_iv: "b6d4d3deb2b929d60a5b35a601ba2a9a"
server_key_version: "encryptionkey@0"
I20260928 14:55:00.017778 32551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "b7b9cba3a6fa41be999febd380d84440"
format_stamp: "Formatted at 2026-09-28 14:55:00 on dist-test-slave-vk40"
server_key: "2368c385be6b4dfe523a38fd22476f7f"
server_key_iv: "b6d4d3deb2b929d60a5b35a601ba2a9a"
server_key_version: "encryptionkey@0"
I20260928 14:55:00.022974 32551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:00.026384 32566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:00.027791 32551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260928 14:55:00.027944 32551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "b7b9cba3a6fa41be999febd380d84440"
format_stamp: "Formatted at 2026-09-28 14:55:00 on dist-test-slave-vk40"
server_key: "2368c385be6b4dfe523a38fd22476f7f"
server_key_iv: "b6d4d3deb2b929d60a5b35a601ba2a9a"
server_key_version: "encryptionkey@0"
I20260928 14:55:00.028131 32551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:00.046937 32551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:00.050879 32551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:00.051151 32551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:00.052073 32551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:00.053442 32551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:00.053515 32551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:00.053638 32551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:00.053694 32551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:00.084100 32551 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:39331
I20260928 14:55:00.084146 32678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:39331 every 8 connection(s)
I20260928 14:55:00.085448 32551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:55:00.089931 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 32551
I20260928 14:55:00.090117 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:55:00.090796 15303 external_mini_cluster.cc:1428] Setting key 0942e9af944167d4781012d7086d4555
Sep 28 14:55:00 dist-test-slave-vk40 krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607300, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:00.105079 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:00.087929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:34229 (local address 127.14.241.254:34579)
0928 14:55:00.088391 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:00.088395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:00.089312 (+ 917us) server_negotiation.cc:407] Connection header received
0928 14:55:00.091236 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:00.091243 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:00.091330 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:00.091427 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:00.093078 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:00.093794 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:00.095382 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:00.095787 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:00.095909 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:00.099347 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:00.099352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:00.099355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:00.099390 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:00.101944 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:00.102730 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:00.102735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:00.102737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:00.102873 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:00.103296 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:00.103300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:00.103302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:00.103573 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:00.103704 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:00.104477 (+ 773us) server_negotiation.cc:299] Negotiation successful
0928 14:55:00.104714 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":228,"threads_started":1}
I20260928 14:55:00.106091 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:00.088451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:34579 (local address 127.14.241.193:34229)
0928 14:55:00.089144 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:00.089191 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:00.090954 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:00.091693 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:00.091702 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:00.092239 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:00.092902 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:00.092922 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:00.093955 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:00.093958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:00.095237 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:00.095250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:00.096032 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:00.096040 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:00.096304 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:00.096953 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:00.096982 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:00.099111 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:00.102104 (+ 2993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:00.102114 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:00.102134 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:00.102596 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:00.102981 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:00.102986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:00.102990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:00.103186 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:00.103698 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:00.103707 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:00.104070 (+ 363us) client_negotiation.cc:769] Sending connection context
0928 14:55:00.104368 (+ 298us) client_negotiation.cc:240] Negotiation successful
0928 14:55:00.104929 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":254,"threads_started":1}
I20260928 14:55:00.107753 32679 heartbeater.cc:347] Connected to a master server at 127.14.241.254:34579
I20260928 14:55:00.108104 32679 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:00.109057 32679 heartbeater.cc:511] Master 127.14.241.254:34579 requested a full tablet report, sending...
I20260928 14:55:00.112115 32490 ts_manager.cc:194] Registered new tserver with Master: b7b9cba3a6fa41be999febd380d84440 (127.14.241.193:39331)
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
I20260928 14:55:00.114282 32490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:34229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:55:00.144588 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34579
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:46805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:00.468272 32685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:00.468655 32685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:00.468750 32685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:00.476087 32685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:00.476215 32685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:00.476373 32685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:55:00.486351 32685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:46805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:00.487607 32685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:00.488881 32685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:00.505759 32691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:00.505784 32690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:00.506119 32693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:00.507349 32685 server_base.cc:1034] running on GCE node
I20260928 14:55:00.508010 32685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:00.509006 32685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:00.510221 32685 hybrid_clock.cc:648] HybridClock initialized: now 1790607300510162 us; error 57 us; skew 500 ppm
I20260928 14:55:00.510533 32685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:00 dist-test-slave-vk40 krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607300, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:00.513588 32685 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:55:00.515019 32685 webserver.cc:460] Webserver started at http://127.14.241.194:44599/ using document root <none> and password file <none>
I20260928 14:55:00.515718 32685 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:00.515832 32685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:00.516170 32685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:00.519187 32685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "c1069912f8af4989a4900336726d45f2"
format_stamp: "Formatted at 2026-09-28 14:55:00 on dist-test-slave-vk40"
server_key: "ace989cec6073be4be889cb011cdd0cf"
server_key_iv: "56c7cbf11dc8acc16185fe9ffd233a15"
server_key_version: "encryptionkey@0"
I20260928 14:55:00.519989 32685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "c1069912f8af4989a4900336726d45f2"
format_stamp: "Formatted at 2026-09-28 14:55:00 on dist-test-slave-vk40"
server_key: "ace989cec6073be4be889cb011cdd0cf"
server_key_iv: "56c7cbf11dc8acc16185fe9ffd233a15"
server_key_version: "encryptionkey@0"
I20260928 14:55:00.525281 32685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:55:00.529095 32700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:00.530478 32685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:55:00.530637 32685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "c1069912f8af4989a4900336726d45f2"
format_stamp: "Formatted at 2026-09-28 14:55:00 on dist-test-slave-vk40"
server_key: "ace989cec6073be4be889cb011cdd0cf"
server_key_iv: "56c7cbf11dc8acc16185fe9ffd233a15"
server_key_version: "encryptionkey@0"
I20260928 14:55:00.530828 32685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:00.549003 32685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:00.553078 32685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:00.553341 32685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:00.554260 32685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:00.555593 32685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:00.555661 32685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:00.555776 32685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:00.555824 32685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:00.584053 32685 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:43545
I20260928 14:55:00.584105 344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:43545 every 8 connection(s)
I20260928 14:55:00.585327 32685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:55:00.587388 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 32685
I20260928 14:55:00.587543 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:55:00.588052 15303 external_mini_cluster.cc:1428] Setting key 86c3a3e4ec2d11ce94a2b69a3be7fae5
Sep 28 14:55:00 dist-test-slave-vk40 krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607300, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:00.605604 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:00.587859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:34547 (local address 127.14.241.254:34579)
0928 14:55:00.588062 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:00.588068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:00.589397 (+ 1329us) server_negotiation.cc:407] Connection header received
0928 14:55:00.591255 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:00.591260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:00.591354 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:00.591436 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:00.593002 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:00.593725 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:00.595279 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:00.595714 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:00.595827 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:00.599694 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:00.599700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:00.599703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:00.599737 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:00.602346 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:00.603159 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:00.603163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:00.603165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:00.603313 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:00.603760 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:00.603764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:00.603766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:00.604045 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:00.604184 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:00.604975 (+ 791us) server_negotiation.cc:299] Negotiation successful
0928 14:55:00.605235 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 14:55:00.606592 347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:00.588454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:34579 (local address 127.14.241.194:34547)
0928 14:55:00.589234 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:00.589286 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:00.590990 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:00.591682 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:00.591691 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:00.592205 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:00.592824 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:00.592852 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:00.593886 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:00.593890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:00.595135 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:00.595148 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:00.595956 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:00.595964 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:00.596178 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:00.596902 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:00.596937 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:00.599467 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:00.602498 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:00.602508 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:00.602526 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:00.603013 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:00.603423 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:00.603428 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:00.603431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:00.603642 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:00.604171 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:00.604178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:00.604598 (+ 420us) client_negotiation.cc:769] Sending connection context
0928 14:55:00.604850 (+ 252us) client_negotiation.cc:240] Negotiation successful
0928 14:55:00.605412 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":319,"threads_started":1}
I20260928 14:55:00.608103 345 heartbeater.cc:347] Connected to a master server at 127.14.241.254:34579
I20260928 14:55:00.608443 345 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:00.609151 345 heartbeater.cc:511] Master 127.14.241.254:34579 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
I20260928 14:55:00.610647 32490 ts_manager.cc:194] Registered new tserver with Master: c1069912f8af4989a4900336726d45f2 (127.14.241.194:43545)
I20260928 14:55:00.611536 32490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:34547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:55:00.639520 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34579
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:46805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:00.960352 350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:00.960670 350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:00.960862 350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:00.968189 350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:00.968333 350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:00.968494 350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:55:00.978278 350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:46805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:00.979555 350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:00.980983 350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:00.994135 355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:00.994097 356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:00.994163 358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:00.994913 350 server_base.cc:1034] running on GCE node
I20260928 14:55:00.995501 350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:00.996424 350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:00.997607 350 hybrid_clock.cc:648] HybridClock initialized: now 1790607300997556 us; error 43 us; skew 500 ppm
I20260928 14:55:00.997927 350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:00 dist-test-slave-vk40 krb5kdc[32445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607300, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:01.000880 350 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:55:01.002262 350 webserver.cc:460] Webserver started at http://127.14.241.195:37523/ using document root <none> and password file <none>
I20260928 14:55:01.002934 350 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:01.003046 350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:01.003342 350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:01.006217 350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "3157e836fa1d4619b338fb6ac5b81193"
format_stamp: "Formatted at 2026-09-28 14:55:00 on dist-test-slave-vk40"
server_key: "3d7d2289cab09f811ae3a6a9380fd98a"
server_key_iv: "e3ab186038139bd108d62b67ba0635f3"
server_key_version: "encryptionkey@0"
I20260928 14:55:01.006978 350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "3157e836fa1d4619b338fb6ac5b81193"
format_stamp: "Formatted at 2026-09-28 14:55:00 on dist-test-slave-vk40"
server_key: "3d7d2289cab09f811ae3a6a9380fd98a"
server_key_iv: "e3ab186038139bd108d62b67ba0635f3"
server_key_version: "encryptionkey@0"
I20260928 14:55:01.012076 350 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 14:55:01.015543 365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:01.016875 350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:55:01.017023 350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "3157e836fa1d4619b338fb6ac5b81193"
format_stamp: "Formatted at 2026-09-28 14:55:00 on dist-test-slave-vk40"
server_key: "3d7d2289cab09f811ae3a6a9380fd98a"
server_key_iv: "e3ab186038139bd108d62b67ba0635f3"
server_key_version: "encryptionkey@0"
I20260928 14:55:01.017197 350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:01.032510 350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:01.036793 350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:01.037048 350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:01.038029 350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:01.039392 350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:01.039462 350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:01.039568 350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:01.039621 350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:01.069315 350 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:44649
I20260928 14:55:01.069367 477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:44649 every 8 connection(s)
I20260928 14:55:01.070621 350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:55:01.072440 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 350
I20260928 14:55:01.072615 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:55:01.073060 15303 external_mini_cluster.cc:1428] Setting key 175708a3e09ab5ab30c98c831225f3a0
Sep 28 14:55:01 dist-test-slave-vk40 krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607300, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:01.090662 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.073227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:58337 (local address 127.14.241.254:34579)
0928 14:55:01.073454 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:01.073461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:01.074790 (+ 1329us) server_negotiation.cc:407] Connection header received
0928 14:55:01.076933 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:01.076937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:01.077028 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:01.077106 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:01.078537 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.079250 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.080819 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.081226 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.081328 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.084773 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:01.084778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:01.084782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:01.084817 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:01.087707 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:01.088599 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:01.088606 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:01.088610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:01.088782 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:01.089221 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:01.089225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:01.089227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:01.089511 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:01.089626 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:01.090229 (+ 603us) server_negotiation.cc:299] Negotiation successful
0928 14:55:01.090408 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 14:55:01.091585 480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.073847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:34579 (local address 127.14.241.195:58337)
0928 14:55:01.074614 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:01.074670 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:01.076685 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:01.077300 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:01.077308 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:01.077809 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:01.078408 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.078425 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.079373 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.079376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.080694 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.080707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.081399 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.081404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.081610 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.082211 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:01.082232 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:01.084556 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:01.087843 (+ 3287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:01.087853 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:01.087869 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:01.088472 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:01.088915 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:01.088919 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:01.088923 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:01.089116 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:01.089606 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:01.089612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:01.089935 (+ 323us) client_negotiation.cc:769] Sending connection context
0928 14:55:01.090141 (+ 206us) client_negotiation.cc:240] Negotiation successful
0928 14:55:01.090532 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":291,"threads_started":1}
I20260928 14:55:01.092819 478 heartbeater.cc:347] Connected to a master server at 127.14.241.254:34579
I20260928 14:55:01.093096 478 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:01.093775 478 heartbeater.cc:511] Master 127.14.241.254:34579 requested a full tablet report, sending...
I20260928 14:55:01.095141 32490 ts_manager.cc:194] Registered new tserver with Master: 3157e836fa1d4619b338fb6ac5b81193 (127.14.241.195:44649)
I20260928 14:55:01.095942 32490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:58337
I20260928 14:55:01.101166 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:55:01.117794 32679 heartbeater.cc:503] Master 127.14.241.254:34579 was elected leader, sending a full tablet report...
I20260928 14:55:01.120610 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.105607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.14.241.254:34579)
0928 14:55:01.105841 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:01.105848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:01.106167 (+ 319us) server_negotiation.cc:407] Connection header received
0928 14:55:01.106291 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:01.106294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:01.106361 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:01.106439 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:01.107754 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.108517 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.109893 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.110289 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.110387 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.113307 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:01.113315 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:01.113320 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:01.113370 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:01.116667 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:01.118238 (+ 1571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:01.118244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:01.118248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:01.118427 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:01.118790 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:01.118795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:01.118799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:01.119238 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:01.119503 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:01.120007 (+ 504us) server_negotiation.cc:299] Negotiation successful
0928 14:55:01.120222 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 14:55:01.124429 32490 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:55:01.131280 32490 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:55:01.153599 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.143521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40188 (local address 127.14.241.194:43545)
0928 14:55:01.144718 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:01.144726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:01.144752 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:55:01.144857 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:01.144865 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:01.145115 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:01.146038 (+ 923us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:01.146265 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.147880 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.150826 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.152349 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.152553 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.152733 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:01.153082 (+ 349us) server_negotiation.cc:299] Negotiation successful
0928 14:55:01.153256 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1057,"thread_start_us":935,"threads_started":1}
I20260928 14:55:01.153671 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.143184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:43545 (local address 127.0.0.1:40188)
0928 14:55:01.143923 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:01.143974 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:01.144156 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:01.145353 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:01.145360 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:01.145388 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:01.145822 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.145835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.148025 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.148030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.150567 (+ 2537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.150587 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.152588 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.152593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.152812 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.152883 (+ 71us) client_negotiation.cc:769] Sending connection context
0928 14:55:01.153017 (+ 134us) client_negotiation.cc:240] Negotiation successful
0928 14:55:01.153212 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"spinlock_wait_cycles":3456,"thread_start_us":335,"threads_started":1}
I20260928 14:55:01.157845 32748 tablet_service.cc:1467] Processing CreateTablet for tablet 96222381b8c948a8a3b6b87ad38446a5 (DEFAULT_TABLE table=test-table [id=3c3319629b3d48f7b920ec411a75baa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:01.158944 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.144124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55820 (local address 127.14.241.193:39331)
0928 14:55:01.151113 (+ 6989us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:01.151122 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:01.151152 (+ 30us) server_negotiation.cc:407] Connection header received
0928 14:55:01.151258 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:01.151267 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:01.151582 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:01.151807 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:01.152462 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.153880 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.156594 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.157935 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.158117 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.158293 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:01.158467 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 14:55:01.158626 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6774,"thread_start_us":6586,"threads_started":1}
I20260928 14:55:01.158957 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.143160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39331 (local address 127.0.0.1:55820)
0928 14:55:01.143922 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:01.143972 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:01.144156 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:01.151784 (+ 7628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:01.151790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:01.151820 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:01.152246 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.152260 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.154024 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.154029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.156409 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.156424 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.158131 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.158137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.158285 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.158317 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 14:55:01.158412 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 14:55:01.158548 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":18,"spinlock_wait_cycles":20480,"thread_start_us":1479,"threads_started":1}
I20260928 14:55:01.159577 32748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96222381b8c948a8a3b6b87ad38446a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:01.160133 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.144374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:44649 (local address 127.0.0.1:34428)
0928 14:55:01.152458 (+ 8084us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:01.152473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:01.152579 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:01.153513 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:01.153517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:01.153541 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:01.153888 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.153900 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.155317 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.155322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.157457 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.157472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.159367 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.159374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.159537 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.159568 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 14:55:01.159660 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 14:55:01.159786 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7923,"mutex_wait_us":30,"thread_start_us":6711,"threads_started":1}
I20260928 14:55:01.160212 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.144763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34428 (local address 127.14.241.195:44649)
0928 14:55:01.151678 (+ 6915us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:01.151686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:01.152696 (+ 1010us) server_negotiation.cc:407] Connection header received
0928 14:55:01.152803 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:01.152811 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:01.153055 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:01.153236 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:01.154101 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.155168 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.157633 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.158994 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.159365 (+ 371us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.159529 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:01.159751 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 14:55:01.159905 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6743,"thread_start_us":7243,"threads_started":1}
I20260928 14:55:01.162971 32614 tablet_service.cc:1467] Processing CreateTablet for tablet 96222381b8c948a8a3b6b87ad38446a5 (DEFAULT_TABLE table=test-table [id=3c3319629b3d48f7b920ec411a75baa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:01.164239 32614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96222381b8c948a8a3b6b87ad38446a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:01.169391 413 tablet_service.cc:1467] Processing CreateTablet for tablet 96222381b8c948a8a3b6b87ad38446a5 (DEFAULT_TABLE table=test-table [id=3c3319629b3d48f7b920ec411a75baa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:01.170951 413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96222381b8c948a8a3b6b87ad38446a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:01.172729 497 tablet_bootstrap.cc:492] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2: Bootstrap starting.
I20260928 14:55:01.176504 498 tablet_bootstrap.cc:492] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440: Bootstrap starting.
I20260928 14:55:01.177054 497 tablet_bootstrap.cc:654] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:01.178167 497 log.cc:824] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:01.180027 498 tablet_bootstrap.cc:654] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:01.180732 497 tablet_bootstrap.cc:492] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2: No bootstrap required, opened a new log
I20260928 14:55:01.180975 497 ts_tablet_manager.cc:1397] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 14:55:01.181273 498 log.cc:824] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:01.184418 498 tablet_bootstrap.cc:492] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440: No bootstrap required, opened a new log
I20260928 14:55:01.184708 498 ts_tablet_manager.cc:1397] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260928 14:55:01.189196 501 tablet_bootstrap.cc:492] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193: Bootstrap starting.
I20260928 14:55:01.188949 497 raft_consensus.cc:348] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1069912f8af4989a4900336726d45f2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43545 } } peers { permanent_uuid: "b7b9cba3a6fa41be999febd380d84440" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39331 } } peers { permanent_uuid: "3157e836fa1d4619b338fb6ac5b81193" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44649 } }
I20260928 14:55:01.189540 497 raft_consensus.cc:374] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:01.189699 497 raft_consensus.cc:729] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1069912f8af4989a4900336726d45f2, State: Initialized, Role: FOLLOWER
I20260928 14:55:01.190511 497 consensus_queue.cc:260] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1069912f8af4989a4900336726d45f2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43545 } } peers { permanent_uuid: "b7b9cba3a6fa41be999febd380d84440" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39331 } } peers { permanent_uuid: "3157e836fa1d4619b338fb6ac5b81193" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44649 } }
I20260928 14:55:01.191648 345 heartbeater.cc:503] Master 127.14.241.254:34579 was elected leader, sending a full tablet report...
I20260928 14:55:01.192893 501 tablet_bootstrap.cc:654] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:01.193171 497 ts_tablet_manager.cc:1428] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260928 14:55:01.194229 498 raft_consensus.cc:348] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1069912f8af4989a4900336726d45f2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43545 } } peers { permanent_uuid: "b7b9cba3a6fa41be999febd380d84440" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39331 } } peers { permanent_uuid: "3157e836fa1d4619b338fb6ac5b81193" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44649 } }
I20260928 14:55:01.194460 501 log.cc:824] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:01.194571 498 raft_consensus.cc:374] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:01.194666 498 raft_consensus.cc:729] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b9cba3a6fa41be999febd380d84440, State: Initialized, Role: FOLLOWER
I20260928 14:55:01.195195 498 consensus_queue.cc:260] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1069912f8af4989a4900336726d45f2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43545 } } peers { permanent_uuid: "b7b9cba3a6fa41be999febd380d84440" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39331 } } peers { permanent_uuid: "3157e836fa1d4619b338fb6ac5b81193" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44649 } }
I20260928 14:55:01.196830 498 ts_tablet_manager.cc:1428] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260928 14:55:01.199677 501 tablet_bootstrap.cc:492] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193: No bootstrap required, opened a new log
I20260928 14:55:01.199944 501 ts_tablet_manager.cc:1397] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260928 14:55:01.207111 501 raft_consensus.cc:348] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1069912f8af4989a4900336726d45f2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43545 } } peers { permanent_uuid: "b7b9cba3a6fa41be999febd380d84440" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39331 } } peers { permanent_uuid: "3157e836fa1d4619b338fb6ac5b81193" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44649 } }
I20260928 14:55:01.207508 501 raft_consensus.cc:374] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:01.207605 501 raft_consensus.cc:729] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3157e836fa1d4619b338fb6ac5b81193, State: Initialized, Role: FOLLOWER
I20260928 14:55:01.208168 501 consensus_queue.cc:260] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1069912f8af4989a4900336726d45f2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43545 } } peers { permanent_uuid: "b7b9cba3a6fa41be999febd380d84440" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39331 } } peers { permanent_uuid: "3157e836fa1d4619b338fb6ac5b81193" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44649 } }
I20260928 14:55:01.209339 478 heartbeater.cc:503] Master 127.14.241.254:34579 was elected leader, sending a full tablet report...
I20260928 14:55:01.209935 501 ts_tablet_manager.cc:1428] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260928 14:55:01.236716 502 raft_consensus.cc:482] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:55:01.237005 502 raft_consensus.cc:504] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1069912f8af4989a4900336726d45f2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43545 } } peers { permanent_uuid: "b7b9cba3a6fa41be999febd380d84440" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39331 } } peers { permanent_uuid: "3157e836fa1d4619b338fb6ac5b81193" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44649 } }
I20260928 14:55:01.238762 502 leader_election.cc:288] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7b9cba3a6fa41be999febd380d84440 (127.14.241.193:39331), 3157e836fa1d4619b338fb6ac5b81193 (127.14.241.195:44649)
I20260928 14:55:01.245836 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.239421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:42183 (local address 127.14.241.195:44649)
0928 14:55:01.239622 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:01.239628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:01.240208 (+ 580us) server_negotiation.cc:407] Connection header received
0928 14:55:01.240386 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:01.240390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:01.240447 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:01.240529 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:01.241169 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.241934 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.243763 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.244875 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.245019 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.245104 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:01.245461 (+ 357us) server_negotiation.cc:299] Negotiation successful
0928 14:55:01.245586 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 14:55:01.245862 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.239161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:44565 (local address 127.14.241.193:39331)
0928 14:55:01.239366 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:01.239373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:01.240109 (+ 736us) server_negotiation.cc:407] Connection header received
0928 14:55:01.240352 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:01.240357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:01.240427 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:01.240493 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:01.241126 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.241875 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.243799 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.244725 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.244883 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.244957 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:01.245554 (+ 597us) server_negotiation.cc:299] Negotiation successful
0928 14:55:01.245670 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 14:55:01.246058 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.239220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:44649 (local address 127.14.241.194:42183)
0928 14:55:01.240088 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:01.240101 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:01.240249 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:01.240555 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:01.240559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:01.240582 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:01.241016 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.241025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.242071 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.242074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.243604 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.243616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.245093 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.245099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.245280 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.245346 (+ 66us) client_negotiation.cc:769] Sending connection context
0928 14:55:01.245431 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:55:01.245562 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":35,"thread_start_us":568,"threads_started":1}
I20260928 14:55:01.246524 506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.239143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39331 (local address 127.14.241.194:44565)
0928 14:55:01.239994 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:01.240013 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:01.240166 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:01.240530 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:01.240535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:01.240557 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:01.240977 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.240989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.241998 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.242003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.243641 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:01.243653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.245123 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.245129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:01.245367 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.245399 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 14:55:01.246049 (+ 650us) client_negotiation.cc:240] Negotiation successful
0928 14:55:01.246180 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":116,"thread_start_us":464,"threads_started":1}
I20260928 14:55:01.247118 433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96222381b8c948a8a3b6b87ad38446a5" candidate_uuid: "c1069912f8af4989a4900336726d45f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3157e836fa1d4619b338fb6ac5b81193" is_pre_election: true
I20260928 14:55:01.247440 32634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96222381b8c948a8a3b6b87ad38446a5" candidate_uuid: "c1069912f8af4989a4900336726d45f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9cba3a6fa41be999febd380d84440" is_pre_election: true
I20260928 14:55:01.247607 433 raft_consensus.cc:2454] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1069912f8af4989a4900336726d45f2 in term 0.
I20260928 14:55:01.247974 32634 raft_consensus.cc:2454] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1069912f8af4989a4900336726d45f2 in term 0.
I20260928 14:55:01.248234 32703 leader_election.cc:302] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3157e836fa1d4619b338fb6ac5b81193, c1069912f8af4989a4900336726d45f2; no voters:
I20260928 14:55:01.248641 502 raft_consensus.cc:2781] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:55:01.248787 502 raft_consensus.cc:482] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:55:01.248883 502 raft_consensus.cc:3037] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:01.251010 502 raft_consensus.cc:504] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1069912f8af4989a4900336726d45f2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43545 } } peers { permanent_uuid: "b7b9cba3a6fa41be999febd380d84440" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39331 } } peers { permanent_uuid: "3157e836fa1d4619b338fb6ac5b81193" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44649 } }
I20260928 14:55:01.251564 502 leader_election.cc:288] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [CANDIDATE]: Term 1 election: Requested vote from peers b7b9cba3a6fa41be999febd380d84440 (127.14.241.193:39331), 3157e836fa1d4619b338fb6ac5b81193 (127.14.241.195:44649)
I20260928 14:55:01.252029 32634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96222381b8c948a8a3b6b87ad38446a5" candidate_uuid: "c1069912f8af4989a4900336726d45f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9cba3a6fa41be999febd380d84440"
I20260928 14:55:01.252075 433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96222381b8c948a8a3b6b87ad38446a5" candidate_uuid: "c1069912f8af4989a4900336726d45f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3157e836fa1d4619b338fb6ac5b81193"
I20260928 14:55:01.252255 32634 raft_consensus.cc:3037] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:01.252255 433 raft_consensus.cc:3037] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:01.255410 433 raft_consensus.cc:2454] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1069912f8af4989a4900336726d45f2 in term 1.
I20260928 14:55:01.255410 32634 raft_consensus.cc:2454] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1069912f8af4989a4900336726d45f2 in term 1.
I20260928 14:55:01.255857 32701 leader_election.cc:302] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7b9cba3a6fa41be999febd380d84440, c1069912f8af4989a4900336726d45f2; no voters:
I20260928 14:55:01.256151 502 raft_consensus.cc:2781] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:01.256601 502 raft_consensus.cc:686] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 1 LEADER]: Becoming Leader. State: Replica: c1069912f8af4989a4900336726d45f2, State: Running, Role: LEADER
I20260928 14:55:01.257076 502 consensus_queue.cc:237] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1069912f8af4989a4900336726d45f2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43545 } } peers { permanent_uuid: "b7b9cba3a6fa41be999febd380d84440" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39331 } } peers { permanent_uuid: "3157e836fa1d4619b338fb6ac5b81193" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44649 } }
I20260928 14:55:01.261716 32489 catalog_manager.cc:5495] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1069912f8af4989a4900336726d45f2 (127.14.241.194). New cstate: current_term: 1 leader_uuid: "c1069912f8af4989a4900336726d45f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1069912f8af4989a4900336726d45f2" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 43545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7b9cba3a6fa41be999febd380d84440" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3157e836fa1d4619b338fb6ac5b81193" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44649 } health_report { overall_health: UNKNOWN } } }
I20260928 14:55:01.280683 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.275776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40200 (local address 127.14.241.194:43545)
0928 14:55:01.275995 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:01.276001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:01.276040 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:55:01.276208 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:01.276212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:01.276326 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:01.276453 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:55:01.276753 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.277450 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.278732 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.279146 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.279237 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.279648 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:55:01.280160 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:55:01.280271 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:01.280353 (+ 82us) server_negotiation.cc:299] Negotiation successful
0928 14:55:01.280425 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:55:01.289335 32634 raft_consensus.cc:1261] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440 [term 1 FOLLOWER]: Refusing update from remote peer c1069912f8af4989a4900336726d45f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:01.289342 433 raft_consensus.cc:1261] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193 [term 1 FOLLOWER]: Refusing update from remote peer c1069912f8af4989a4900336726d45f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:01.290584 502 consensus_queue.cc:1035] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3157e836fa1d4619b338fb6ac5b81193" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 44649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:01.290953 508 consensus_queue.cc:1035] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9cba3a6fa41be999febd380d84440" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:01.302618 511 mvcc.cc:205] Tried to move back new op lower bound from 7334327506069245952 to 7334327505952669696. Current Snapshot: MvccSnapshot[applied={T|T < 7334327506069245952}]
I20260928 14:55:01.321025 32490 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:01.321342 32490 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:01.325189 32490 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c002bbb4203140289dad6c55f13ee0c6: Sending DeleteTablet for 3 replicas of tablet 96222381b8c948a8a3b6b87ad38446a5
I20260928 14:55:01.326504 32614 tablet_service.cc:1514] Processing DeleteTablet for tablet 96222381b8c948a8a3b6b87ad38446a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:01 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:55820
I20260928 14:55:01.326706 413 tablet_service.cc:1514] Processing DeleteTablet for tablet 96222381b8c948a8a3b6b87ad38446a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:01 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:34428
I20260928 14:55:01.327237 32748 tablet_service.cc:1514] Processing DeleteTablet for tablet 96222381b8c948a8a3b6b87ad38446a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:01 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:40188
I20260928 14:55:01.328658 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 32551
I20260928 14:55:01.344610 524 tablet_replica.cc:330] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2: stopping tablet replica
I20260928 14:55:01.347724 522 tablet_replica.cc:330] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440: stopping tablet replica
I20260928 14:55:01.348374 522 raft_consensus.cc:2229] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:01.348397 524 raft_consensus.cc:2229] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:55:01.349030 522 raft_consensus.cc:2258] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:01.349196 524 raft_consensus.cc:2258] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:01.349797 523 tablet_replica.cc:330] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193: stopping tablet replica
I20260928 14:55:01.350119 523 raft_consensus.cc:2229] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:01.350513 523 raft_consensus.cc:2258] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:01.351346 522 ts_tablet_manager.cc:1905] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:01.353092 524 ts_tablet_manager.cc:1905] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:01.358668 522 ts_tablet_manager.cc:1918] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:01.358960 522 log.cc:1196] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/96222381b8c948a8a3b6b87ad38446a5
I20260928 14:55:01.359452 524 ts_tablet_manager.cc:1918] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:01.359581 522 ts_tablet_manager.cc:1939] T 96222381b8c948a8a3b6b87ad38446a5 P b7b9cba3a6fa41be999febd380d84440: Deleting consensus metadata
I20260928 14:55:01.359714 524 log.cc:1196] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/96222381b8c948a8a3b6b87ad38446a5
Sep 28 14:55:01 dist-test-slave-vk40 krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:55:01.361104 524 ts_tablet_manager.cc:1939] T 96222381b8c948a8a3b6b87ad38446a5 P c1069912f8af4989a4900336726d45f2: Deleting consensus metadata
I20260928 14:55:01.361660 523 ts_tablet_manager.cc:1905] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:01.361685 32474 catalog_manager.cc:4841] TS b7b9cba3a6fa41be999febd380d84440 (127.14.241.193:39331): tablet 96222381b8c948a8a3b6b87ad38446a5 (table test-table [id=3c3319629b3d48f7b920ec411a75baa8]) successfully deleted
I20260928 14:55:01.362973 32476 catalog_manager.cc:4841] TS c1069912f8af4989a4900336726d45f2 (127.14.241.194:43545): tablet 96222381b8c948a8a3b6b87ad38446a5 (table test-table [id=3c3319629b3d48f7b920ec411a75baa8]) successfully deleted
I20260928 14:55:01.368126 523 ts_tablet_manager.cc:1918] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:01.368384 523 log.cc:1196] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/96222381b8c948a8a3b6b87ad38446a5
I20260928 14:55:01.368884 523 ts_tablet_manager.cc:1939] T 96222381b8c948a8a3b6b87ad38446a5 P 3157e836fa1d4619b338fb6ac5b81193: Deleting consensus metadata
I20260928 14:55:01.370702 32476 catalog_manager.cc:4841] TS 3157e836fa1d4619b338fb6ac5b81193 (127.14.241.195:44649): tablet 96222381b8c948a8a3b6b87ad38446a5 (table test-table [id=3c3319629b3d48f7b920ec411a75baa8]) successfully deleted
I20260928 14:55:01.370802 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.329599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55834 (local address 127.14.241.193:39331)
0928 14:55:01.329778 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:01.329784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:01.341376 (+ 11592us) server_negotiation.cc:407] Connection header received
0928 14:55:01.341903 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:01.341910 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:01.342004 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:01.342081 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:01.344902 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.346218 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.351362 (+ 5144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.351994 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.352127 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.363526 (+ 11399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:01.363542 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:01.363561 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:01.363611 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:01.367283 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:01.368319 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:01.368337 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:01.368341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:01.368519 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:01.368971 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:01.368976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:01.368979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:01.369470 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:01.369778 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:01.370097 (+ 319us) server_negotiation.cc:299] Negotiation successful
0928 14:55:01.370351 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 14:55:01.371505 32674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:01.460770 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 32551
I20260928 14:55:01.476167 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 32685
Sep 28 14:55:01 dist-test-slave-vk40 krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:55:01.493062 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.476960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40202 (local address 127.14.241.194:43545)
0928 14:55:01.477156 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:01.477162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:01.477186 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:55:01.477305 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:01.477314 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:01.477398 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:01.477505 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:01.478692 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.479913 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.481203 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.481871 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.482000 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.486258 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:01.486273 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:01.486292 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:01.486346 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:01.490292 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:01.490953 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:01.490964 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:01.490968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:01.491136 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:01.491459 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:01.491464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:01.491470 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:01.491930 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:01.492266 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:01.492459 (+ 193us) server_negotiation.cc:299] Negotiation successful
0928 14:55:01.492681 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 14:55:01.493806 340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:01.607450 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 32685
I20260928 14:55:01.622792 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 350
Sep 28 14:55:01 dist-test-slave-vk40 krb5kdc[32445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:55:01.637171 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:01.623487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34436 (local address 127.14.241.195:44649)
0928 14:55:01.623646 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:01.623650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:01.623688 (+ 38us) server_negotiation.cc:407] Connection header received
0928 14:55:01.623821 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:01.623825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:01.623900 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:01.623983 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:01.625337 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.626075 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.627398 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:01.627786 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:01.627879 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:01.632036 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:01.632053 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:01.632069 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:01.632105 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:01.634538 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:01.635168 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:01.635175 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:01.635178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:01.635393 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:01.635720 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:01.635723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:01.635725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:01.636077 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:01.636356 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:01.636757 (+ 401us) server_negotiation.cc:299] Negotiation successful
0928 14:55:01.636931 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 14:55:01.637800 473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:01.725229 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 350
I20260928 14:55:01.739187 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 32458
I20260928 14:55:01.740355 32520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:01.860877 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 32458
2026-09-28T14:55:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4055 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-28T14:55:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:01Z Disabled control of system clock
I20260928 14:55:01.936326 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:37535
--webserver_interface=127.14.241.254
--webserver_port=0
--builtin_ntp_servers=127.14.241.212:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:37535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:55:02.257412 533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:02.257704 533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:02.257784 533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:02.265233 533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:02.265348 533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:02.265384 533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:02.265443 533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:02.265486 533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:02.274813 533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:37535
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:37535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:02.276055 533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:02.277891 533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:02.288968 538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:02.289013 539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:02.289139 541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:02.290210 533 server_base.cc:1034] running on GCE node
I20260928 14:55:02.290982 533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:02.292380 533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:02.293588 533 hybrid_clock.cc:648] HybridClock initialized: now 1790607302293534 us; error 46 us; skew 500 ppm
I20260928 14:55:02.293926 533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:02.296044 533 webserver.cc:460] Webserver started at http://127.14.241.254:46581/ using document root <none> and password file <none>
I20260928 14:55:02.296723 533 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:02.296821 533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:02.297124 533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:02.300042 533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "64a17e1ee96442ccbc8e0507e0226639"
format_stamp: "Formatted at 2026-09-28 14:55:02 on dist-test-slave-vk40"
server_key: "e00146888b0eb6cc0631999a9a2c9415"
server_key_iv: "83bdbba8d98b027957d15f9d1282fbdc"
server_key_version: "encryptionkey@0"
I20260928 14:55:02.300886 533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "64a17e1ee96442ccbc8e0507e0226639"
format_stamp: "Formatted at 2026-09-28 14:55:02 on dist-test-slave-vk40"
server_key: "e00146888b0eb6cc0631999a9a2c9415"
server_key_iv: "83bdbba8d98b027957d15f9d1282fbdc"
server_key_version: "encryptionkey@0"
I20260928 14:55:02.306157 533 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:02.309837 547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:02.311193 533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:55:02.311358 533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "64a17e1ee96442ccbc8e0507e0226639"
format_stamp: "Formatted at 2026-09-28 14:55:02 on dist-test-slave-vk40"
server_key: "e00146888b0eb6cc0631999a9a2c9415"
server_key_iv: "83bdbba8d98b027957d15f9d1282fbdc"
server_key_version: "encryptionkey@0"
I20260928 14:55:02.311566 533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:02.333788 533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:02.334650 533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:02.334923 533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:02.352348 533 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:37535
I20260928 14:55:02.352370 598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:37535 every 8 connection(s)
I20260928 14:55:02.353701 533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:02.357733 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 533
I20260928 14:55:02.357848 599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:02.357916 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:02.358482 15303 external_mini_cluster.cc:1428] Setting key ca2b6ca2a1249ce62c1bb3b0b006be3f
I20260928 14:55:02.370987 599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639: Bootstrap starting.
I20260928 14:55:02.371775 602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:02.360516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53924 (local address 127.14.241.254:37535)
0928 14:55:02.361594 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:02.361606 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:02.361647 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:55:02.363156 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:02.363177 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:02.363705 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:02.364094 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:02.364670 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:02.365970 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:02.367349 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:02.368058 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:02.368245 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:02.369430 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:02.369438 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:02.369452 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:02.369996 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:02.370077 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:02.370098 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:02.370195 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:02.370408 (+ 213us) server_negotiation.cc:299] Negotiation successful
0928 14:55:02.370723 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":409,"threads_started":1}
I20260928 14:55:02.374167 599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:02.375232 599 log.cc:824] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:02.377903 599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639: No bootstrap required, opened a new log
I20260928 14:55:02.385131 599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a17e1ee96442ccbc8e0507e0226639" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 37535 } }
I20260928 14:55:02.385593 599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:02.385695 599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64a17e1ee96442ccbc8e0507e0226639, State: Initialized, Role: FOLLOWER
I20260928 14:55:02.386258 599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a17e1ee96442ccbc8e0507e0226639" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 37535 } }
I20260928 14:55:02.386480 599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:02.386579 599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:02.386708 599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:02.388835 599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a17e1ee96442ccbc8e0507e0226639" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 37535 } }
I20260928 14:55:02.389277 599 leader_election.cc:302] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64a17e1ee96442ccbc8e0507e0226639; no voters:
I20260928 14:55:02.389791 599 leader_election.cc:288] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:02.389947 604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:02.390619 604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [term 1 LEADER]: Becoming Leader. State: Replica: 64a17e1ee96442ccbc8e0507e0226639, State: Running, Role: LEADER
I20260928 14:55:02.391120 604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a17e1ee96442ccbc8e0507e0226639" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 37535 } }
I20260928 14:55:02.391723 599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:02.393803 606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64a17e1ee96442ccbc8e0507e0226639. Latest consensus state: current_term: 1 leader_uuid: "64a17e1ee96442ccbc8e0507e0226639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a17e1ee96442ccbc8e0507e0226639" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 37535 } } }
I20260928 14:55:02.394193 606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:02.393885 605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64a17e1ee96442ccbc8e0507e0226639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a17e1ee96442ccbc8e0507e0226639" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 37535 } } }
I20260928 14:55:02.395166 605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:02.396219 609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:02.403748 609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:02.413677 609 catalog_manager.cc:1347] Generated new cluster ID: d2e5944995b04218ba718d6dd3353ff8
I20260928 14:55:02.413805 609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:02.426497 609 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:02.428375 609 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:02.438391 609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639: Generated new TSK 0
I20260928 14:55:02.439247 609 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:02.452080 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:37535
--builtin_ntp_servers=127.14.241.212:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:55:02.803416 623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:02.803683 623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:02.803807 623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:02.811070 623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:02.811170 623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:02.811319 623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:55:02.820544 623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:37535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:02.821794 623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:02.823580 623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:02.836180 631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:02.836208 628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:02.836268 629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:02.836934 623 server_base.cc:1034] running on GCE node
I20260928 14:55:02.837548 623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:02.838500 623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:02.839711 623 hybrid_clock.cc:648] HybridClock initialized: now 1790607302839656 us; error 57 us; skew 500 ppm
I20260928 14:55:02.840014 623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:02.847362 623 webserver.cc:460] Webserver started at http://127.14.241.193:41017/ using document root <none> and password file <none>
I20260928 14:55:02.848055 623 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:02.848150 623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:02.848482 623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:02.851301 623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "64bd8c147c8249e3bd2a7977fee21ff2"
format_stamp: "Formatted at 2026-09-28 14:55:02 on dist-test-slave-vk40"
server_key: "01be60555b6db40261d4291735875c36"
server_key_iv: "6abf52ba9accbfc936872c0f033a23f1"
server_key_version: "encryptionkey@0"
I20260928 14:55:02.852046 623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "64bd8c147c8249e3bd2a7977fee21ff2"
format_stamp: "Formatted at 2026-09-28 14:55:02 on dist-test-slave-vk40"
server_key: "01be60555b6db40261d4291735875c36"
server_key_iv: "6abf52ba9accbfc936872c0f033a23f1"
server_key_version: "encryptionkey@0"
I20260928 14:55:02.856913 623 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:55:02.860149 637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:02.861443 623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 14:55:02.861586 623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "64bd8c147c8249e3bd2a7977fee21ff2"
format_stamp: "Formatted at 2026-09-28 14:55:02 on dist-test-slave-vk40"
server_key: "01be60555b6db40261d4291735875c36"
server_key_iv: "6abf52ba9accbfc936872c0f033a23f1"
server_key_version: "encryptionkey@0"
I20260928 14:55:02.861778 623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:02.898401 623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:02.899243 623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:02.899518 623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:02.900537 623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:02.901975 623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:02.902061 623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:02.902165 623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:02.902236 623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:02.931304 623 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:40519
I20260928 14:55:02.931373 751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:40519 every 8 connection(s)
I20260928 14:55:02.932708 623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:55:02.939781 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 623
I20260928 14:55:02.939924 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:55:02.940389 15303 external_mini_cluster.cc:1428] Setting key 2b944a7f71479e284bfe033d1fad761c
I20260928 14:55:02.943006 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:37535
--builtin_ntp_servers=127.14.241.212:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:55:02.945546 754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:02.935474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:54045 (local address 127.14.241.254:37535)
0928 14:55:02.935901 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:02.935905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:02.937187 (+ 1282us) server_negotiation.cc:407] Connection header received
0928 14:55:02.938910 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:02.938915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:02.938982 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:02.939076 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:02.940139 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:02.940993 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:02.942809 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:02.943286 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:02.943403 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:02.944737 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:02.944741 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:02.944744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:02.944763 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:02.944801 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:02.944813 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:02.944889 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:02.945096 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 14:55:02.945195 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":196,"threads_started":1}
I20260928 14:55:02.946556 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:02.936115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:37535 (local address 127.14.241.193:54045)
0928 14:55:02.937030 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:02.937074 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:02.938658 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:02.939240 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:02.939248 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:02.939338 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:02.939952 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:02.939979 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:02.941150 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:02.941156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:02.942670 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:02.942683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:02.943509 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:02.943516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:02.943751 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:02.944429 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:55:02.944431 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:02.944471 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:55:02.944555 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:02.944900 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:02.944911 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:02.944919 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 14:55:02.945074 (+ 155us) client_negotiation.cc:240] Negotiation successful
0928 14:55:02.945473 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":310,"threads_started":1}
I20260928 14:55:02.948122 752 heartbeater.cc:347] Connected to a master server at 127.14.241.254:37535
I20260928 14:55:02.948447 752 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:02.949205 752 heartbeater.cc:511] Master 127.14.241.254:37535 requested a full tablet report, sending...
I20260928 14:55:02.951263 564 ts_manager.cc:194] Registered new tserver with Master: 64bd8c147c8249e3bd2a7977fee21ff2 (127.14.241.193:40519)
I20260928 14:55:02.952725 564 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.193:54045
W20260928 14:55:03.266301 756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:03.266575 756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:03.266700 756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:03.273952 756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:03.274049 756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:03.274184 756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:55:03.283284 756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:37535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:03.284565 756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:03.286525 756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:03.299361 761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:03.299522 764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:03.299393 762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:03.300127 756 server_base.cc:1034] running on GCE node
I20260928 14:55:03.300753 756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:03.301723 756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:03.302933 756 hybrid_clock.cc:648] HybridClock initialized: now 1790607303302882 us; error 45 us; skew 500 ppm
I20260928 14:55:03.303277 756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:03.305490 756 webserver.cc:460] Webserver started at http://127.14.241.194:45159/ using document root <none> and password file <none>
I20260928 14:55:03.306188 756 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:03.306288 756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:03.306586 756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:03.309588 756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "6ea757cb41894242b4c3cf7b04b157d9"
format_stamp: "Formatted at 2026-09-28 14:55:03 on dist-test-slave-vk40"
server_key: "139409baeb0da6f8e40918a879c11eed"
server_key_iv: "4b286314aad7e083a0bca4112148681c"
server_key_version: "encryptionkey@0"
I20260928 14:55:03.310384 756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "6ea757cb41894242b4c3cf7b04b157d9"
format_stamp: "Formatted at 2026-09-28 14:55:03 on dist-test-slave-vk40"
server_key: "139409baeb0da6f8e40918a879c11eed"
server_key_iv: "4b286314aad7e083a0bca4112148681c"
server_key_version: "encryptionkey@0"
I20260928 14:55:03.315611 756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:55:03.319136 770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:03.320461 756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260928 14:55:03.320608 756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "6ea757cb41894242b4c3cf7b04b157d9"
format_stamp: "Formatted at 2026-09-28 14:55:03 on dist-test-slave-vk40"
server_key: "139409baeb0da6f8e40918a879c11eed"
server_key_iv: "4b286314aad7e083a0bca4112148681c"
server_key_version: "encryptionkey@0"
I20260928 14:55:03.320806 756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:03.359688 756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:03.360608 756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:03.360898 756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:03.361850 756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:03.363292 756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:03.363390 756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:03.363492 756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:03.363559 756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:03.393544 756 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:34935
I20260928 14:55:03.393592 883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:34935 every 8 connection(s)
I20260928 14:55:03.394870 756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:55:03.405179 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 756
I20260928 14:55:03.405407 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:55:03.405922 15303 external_mini_cluster.cc:1428] Setting key 39be2390c1278cd2ce23328253eb34c7
I20260928 14:55:03.407155 754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:03.397350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:45779 (local address 127.14.241.254:37535)
0928 14:55:03.397566 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:03.397572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:03.398968 (+ 1396us) server_negotiation.cc:407] Connection header received
0928 14:55:03.400740 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:03.400744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:03.400813 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:03.400907 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:03.401950 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.402677 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.404229 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.404669 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.404767 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:03.406311 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:03.406316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:03.406319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:03.406339 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:03.406375 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:03.406386 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:03.406450 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:03.406719 (+ 269us) server_negotiation.cc:299] Negotiation successful
0928 14:55:03.406835 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:55:03.408349 886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:03.397932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:37535 (local address 127.14.241.194:45779)
0928 14:55:03.398802 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:03.398852 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:03.400509 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:03.401095 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:03.401104 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:03.401195 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:03.401778 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:03.401796 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.402806 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.402809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:03.404091 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:03.404104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.404890 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.404898 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:03.405201 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:03.405935 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:55:03.405939 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:03.405993 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:55:03.406103 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:03.406481 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:03.406494 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:03.406502 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 14:55:03.406685 (+ 183us) client_negotiation.cc:240] Negotiation successful
0928 14:55:03.407112 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":255,"threads_started":1}
I20260928 14:55:03.409035 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:37535
--builtin_ntp_servers=127.14.241.212:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:55:03.409760 884 heartbeater.cc:347] Connected to a master server at 127.14.241.254:37535
I20260928 14:55:03.410091 884 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:03.410975 884 heartbeater.cc:511] Master 127.14.241.254:37535 requested a full tablet report, sending...
I20260928 14:55:03.412504 564 ts_manager.cc:194] Registered new tserver with Master: 6ea757cb41894242b4c3cf7b04b157d9 (127.14.241.194:34935)
I20260928 14:55:03.413311 564 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.194:45779
W20260928 14:55:03.727262 887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:03.727558 887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:03.727679 887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:03.734848 887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:03.734944 887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:03.735087 887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:55:03.744168 887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:37535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:03.745520 887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:03.747253 887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:03.759483 892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:03.760020 893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:03.760064 895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:03.760252 887 server_base.cc:1034] running on GCE node
I20260928 14:55:03.760851 887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:03.761718 887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:03.762924 887 hybrid_clock.cc:648] HybridClock initialized: now 1790607303762873 us; error 46 us; skew 500 ppm
I20260928 14:55:03.763221 887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:03.765360 887 webserver.cc:460] Webserver started at http://127.14.241.195:46637/ using document root <none> and password file <none>
I20260928 14:55:03.766147 887 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:03.766249 887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:03.766535 887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:03.769309 887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "021cdb23d99e4b84a7515be70e160cf9"
format_stamp: "Formatted at 2026-09-28 14:55:03 on dist-test-slave-vk40"
server_key: "cd2f79b29c21a484ffb3ebdacb7cd1fb"
server_key_iv: "9cf6b95aea51f31cac8a41477828fd0b"
server_key_version: "encryptionkey@0"
I20260928 14:55:03.770045 887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "021cdb23d99e4b84a7515be70e160cf9"
format_stamp: "Formatted at 2026-09-28 14:55:03 on dist-test-slave-vk40"
server_key: "cd2f79b29c21a484ffb3ebdacb7cd1fb"
server_key_iv: "9cf6b95aea51f31cac8a41477828fd0b"
server_key_version: "encryptionkey@0"
I20260928 14:55:03.774911 887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260928 14:55:03.778180 901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:03.779408 887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:55:03.779552 887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "021cdb23d99e4b84a7515be70e160cf9"
format_stamp: "Formatted at 2026-09-28 14:55:03 on dist-test-slave-vk40"
server_key: "cd2f79b29c21a484ffb3ebdacb7cd1fb"
server_key_iv: "9cf6b95aea51f31cac8a41477828fd0b"
server_key_version: "encryptionkey@0"
I20260928 14:55:03.779737 887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:03.794196 887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:03.794962 887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:03.795231 887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:03.796099 887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:03.797559 887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:03.797652 887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:03.797755 887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:03.797827 887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:03.826876 887 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:33401
I20260928 14:55:03.826920 1014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:33401 every 8 connection(s)
I20260928 14:55:03.828126 887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:55:03.833264 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 887
I20260928 14:55:03.833448 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:55:03.833987 15303 external_mini_cluster.cc:1428] Setting key e7055398b60b8eaed599c1f0e156fbd1
I20260928 14:55:03.840547 754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:03.830494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:54067 (local address 127.14.241.254:37535)
0928 14:55:03.830653 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:03.830658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:03.831992 (+ 1334us) server_negotiation.cc:407] Connection header received
0928 14:55:03.834205 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:03.834211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:03.834285 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:03.834378 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:03.835649 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.836704 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.838202 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.838570 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.838652 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:03.839826 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:03.839830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:03.839835 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:03.839852 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:03.839877 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:03.839887 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:03.839945 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:03.840141 (+ 196us) server_negotiation.cc:299] Negotiation successful
0928 14:55:03.840226 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:55:03.841485 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:03.830994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:37535 (local address 127.14.241.195:54067)
0928 14:55:03.831838 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:03.831874 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:03.833863 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:03.834593 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:03.834603 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:03.834728 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:03.835485 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:03.835512 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.836877 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.836881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:03.838097 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:03.838109 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.838721 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.838726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:03.838938 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:03.839558 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:55:03.839561 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:03.839600 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:55:03.839673 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:03.839968 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:03.839978 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:03.839987 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 14:55:03.840125 (+ 138us) client_negotiation.cc:240] Negotiation successful
0928 14:55:03.840481 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":255,"threads_started":1}
I20260928 14:55:03.842664 1015 heartbeater.cc:347] Connected to a master server at 127.14.241.254:37535
I20260928 14:55:03.842914 1015 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:03.843580 1015 heartbeater.cc:511] Master 127.14.241.254:37535 requested a full tablet report, sending...
I20260928 14:55:03.844857 564 ts_manager.cc:194] Registered new tserver with Master: 021cdb23d99e4b84a7515be70e160cf9 (127.14.241.195:33401)
I20260928 14:55:03.845575 564 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.195:54067
I20260928 14:55:03.848894 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:55:03.858376 754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:03.852667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51476 (local address 127.14.241.254:37535)
0928 14:55:03.852881 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:03.852888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:03.853199 (+ 311us) server_negotiation.cc:407] Connection header received
0928 14:55:03.853387 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:03.853391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:03.853463 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:03.853540 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:03.854009 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.854694 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.856062 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.856499 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.856606 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:03.857688 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:03.857693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:03.857697 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:03.857718 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:03.857750 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:03.857762 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:03.857835 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:03.857987 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 14:55:03.858082 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:55:03.861874 564 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:55:03.868666 564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:55:03.891386 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:03.880114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:33401 (local address 127.0.0.1:60998)
0928 14:55:03.880861 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:03.880911 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:03.881081 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:03.882355 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:03.882363 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:03.882397 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:03.882903 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:03.882927 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.885168 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.885175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:03.887810 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:03.887837 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.890297 (+ 2460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.890305 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:03.890522 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:03.890576 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 14:55:03.890713 (+ 137us) client_negotiation.cc:240] Negotiation successful
0928 14:55:03.890952 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":346,"threads_started":1}
I20260928 14:55:03.893095 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:03.880373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.14.241.195:33401)
0928 14:55:03.881710 (+ 1337us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:03.881720 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:03.881756 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:55:03.881867 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:03.881875 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:03.882118 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:03.882323 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:03.883133 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.884990 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.888087 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.889688 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.889902 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:03.890152 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:03.891869 (+ 1717us) server_negotiation.cc:299] Negotiation successful
0928 14:55:03.892082 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1185,"thread_start_us":1034,"threads_started":1}
I20260928 14:55:03.896685 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:03.880798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43334 (local address 127.14.241.194:34935)
0928 14:55:03.882094 (+ 1296us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:03.882102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:03.882972 (+ 870us) server_negotiation.cc:407] Connection header received
0928 14:55:03.883270 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:03.883280 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:03.883557 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:03.883713 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:03.886100 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.889037 (+ 2937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.891768 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.895002 (+ 3234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.895177 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:03.895416 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:03.896038 (+ 622us) server_negotiation.cc:299] Negotiation successful
0928 14:55:03.896222 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1150,"thread_start_us":953,"threads_started":1}
I20260928 14:55:03.898633 1028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:03.880167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:34935 (local address 127.0.0.1:43334)
0928 14:55:03.882865 (+ 2698us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:03.882886 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:03.883076 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:03.885383 (+ 2307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:03.885389 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:03.885421 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:03.885855 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:03.885867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.889234 (+ 3367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.889240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:03.891530 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:03.891545 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.895611 (+ 4066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.895618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:03.895832 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:03.895866 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 14:55:03.896894 (+ 1028us) client_negotiation.cc:240] Negotiation successful
0928 14:55:03.897049 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2475,"thread_start_us":2272,"threads_started":1}
I20260928 14:55:03.898466 949 tablet_service.cc:1467] Processing CreateTablet for tablet c3cec26dc92941068431b4b8ddc08f22 (DEFAULT_TABLE table=test-table [id=a070c1332ec3458a80799a715e535832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:03.900166 949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3cec26dc92941068431b4b8ddc08f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:03.902442 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:03.882627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:40519 (local address 127.0.0.1:55292)
0928 14:55:03.892622 (+ 9995us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:03.892643 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:03.892758 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:03.895977 (+ 3219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:03.895981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:03.896012 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:03.896203 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:03.896215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.898298 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.898303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:03.899795 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:03.899811 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.901628 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.901634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:03.901819 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:03.901853 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 14:55:03.901958 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 14:55:03.902104 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9853,"mutex_wait_us":56,"thread_start_us":17046,"threads_started":1}
I20260928 14:55:03.902506 1032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:03.882912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55292 (local address 127.14.241.193:40519)
0928 14:55:03.895110 (+ 12198us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:03.895119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:03.895152 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:55:03.895265 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:03.895274 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:03.895544 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:03.896512 (+ 968us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:03.896746 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.898131 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.900025 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:03.901435 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:03.901608 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:03.901805 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:03.902010 (+ 205us) server_negotiation.cc:299] Negotiation successful
0928 14:55:03.902177 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12017,"thread_start_us":12789,"threads_started":1}
I20260928 14:55:03.902688 818 tablet_service.cc:1467] Processing CreateTablet for tablet c3cec26dc92941068431b4b8ddc08f22 (DEFAULT_TABLE table=test-table [id=a070c1332ec3458a80799a715e535832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:03.904177 818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3cec26dc92941068431b4b8ddc08f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:03.906679 686 tablet_service.cc:1467] Processing CreateTablet for tablet c3cec26dc92941068431b4b8ddc08f22 (DEFAULT_TABLE table=test-table [id=a070c1332ec3458a80799a715e535832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:03.908109 686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3cec26dc92941068431b4b8ddc08f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:03.914776 1033 tablet_bootstrap.cc:492] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: Bootstrap starting.
I20260928 14:55:03.918941 1033 tablet_bootstrap.cc:654] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:03.919847 1034 tablet_bootstrap.cc:492] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9: Bootstrap starting.
I20260928 14:55:03.920928 1033 log.cc:824] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:03.922398 1036 tablet_bootstrap.cc:492] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2: Bootstrap starting.
I20260928 14:55:03.923408 1034 tablet_bootstrap.cc:654] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:03.924141 1033 tablet_bootstrap.cc:492] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: No bootstrap required, opened a new log
I20260928 14:55:03.924450 1033 ts_tablet_manager.cc:1397] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260928 14:55:03.924494 1034 log.cc:824] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:03.926014 1036 tablet_bootstrap.cc:654] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:03.927095 1034 tablet_bootstrap.cc:492] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9: No bootstrap required, opened a new log
I20260928 14:55:03.927192 1036 log.cc:824] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:03.927335 1034 ts_tablet_manager.cc:1397] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260928 14:55:03.930213 1036 tablet_bootstrap.cc:492] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2: No bootstrap required, opened a new log
I20260928 14:55:03.930490 1036 ts_tablet_manager.cc:1397] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260928 14:55:03.935209 1034 raft_consensus.cc:348] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021cdb23d99e4b84a7515be70e160cf9" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 33401 } } peers { permanent_uuid: "6ea757cb41894242b4c3cf7b04b157d9" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34935 } } peers { permanent_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40519 } }
I20260928 14:55:03.935647 1034 raft_consensus.cc:374] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:03.935731 1034 raft_consensus.cc:729] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ea757cb41894242b4c3cf7b04b157d9, State: Initialized, Role: FOLLOWER
I20260928 14:55:03.935971 1033 raft_consensus.cc:348] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021cdb23d99e4b84a7515be70e160cf9" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 33401 } } peers { permanent_uuid: "6ea757cb41894242b4c3cf7b04b157d9" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34935 } } peers { permanent_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40519 } }
I20260928 14:55:03.936406 1033 raft_consensus.cc:374] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:03.936493 1033 raft_consensus.cc:729] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021cdb23d99e4b84a7515be70e160cf9, State: Initialized, Role: FOLLOWER
I20260928 14:55:03.936507 1034 consensus_queue.cc:260] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021cdb23d99e4b84a7515be70e160cf9" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 33401 } } peers { permanent_uuid: "6ea757cb41894242b4c3cf7b04b157d9" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34935 } } peers { permanent_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40519 } }
I20260928 14:55:03.937194 1033 consensus_queue.cc:260] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021cdb23d99e4b84a7515be70e160cf9" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 33401 } } peers { permanent_uuid: "6ea757cb41894242b4c3cf7b04b157d9" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34935 } } peers { permanent_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40519 } }
I20260928 14:55:03.945364 1036 raft_consensus.cc:348] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021cdb23d99e4b84a7515be70e160cf9" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 33401 } } peers { permanent_uuid: "6ea757cb41894242b4c3cf7b04b157d9" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34935 } } peers { permanent_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40519 } }
I20260928 14:55:03.945892 1036 raft_consensus.cc:374] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:03.946004 1036 raft_consensus.cc:729] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64bd8c147c8249e3bd2a7977fee21ff2, State: Initialized, Role: FOLLOWER
I20260928 14:55:03.946852 1036 consensus_queue.cc:260] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021cdb23d99e4b84a7515be70e160cf9" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 33401 } } peers { permanent_uuid: "6ea757cb41894242b4c3cf7b04b157d9" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34935 } } peers { permanent_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40519 } }
I20260928 14:55:03.948599 1015 heartbeater.cc:503] Master 127.14.241.254:37535 was elected leader, sending a full tablet report...
I20260928 14:55:03.948613 884 heartbeater.cc:503] Master 127.14.241.254:37535 was elected leader, sending a full tablet report...
I20260928 14:55:03.949306 1033 ts_tablet_manager.cc:1428] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: Time spent starting tablet: real 0.025s user 0.016s sys 0.005s
I20260928 14:55:03.950220 752 heartbeater.cc:503] Master 127.14.241.254:37535 was elected leader, sending a full tablet report...
I20260928 14:55:03.951913 1034 ts_tablet_manager.cc:1428] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9: Time spent starting tablet: real 0.024s user 0.013s sys 0.010s
I20260928 14:55:03.952419 1036 ts_tablet_manager.cc:1428] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2: Time spent starting tablet: real 0.022s user 0.011s sys 0.005s
I20260928 14:55:04.022578 1040 raft_consensus.cc:482] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:55:04.022850 1040 raft_consensus.cc:504] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021cdb23d99e4b84a7515be70e160cf9" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 33401 } } peers { permanent_uuid: "6ea757cb41894242b4c3cf7b04b157d9" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34935 } } peers { permanent_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40519 } }
I20260928 14:55:04.024570 1040 leader_election.cc:288] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ea757cb41894242b4c3cf7b04b157d9 (127.14.241.194:34935), 64bd8c147c8249e3bd2a7977fee21ff2 (127.14.241.193:40519)
I20260928 14:55:04.030767 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:04.024810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:34935 (local address 127.14.241.195:33481)
0928 14:55:04.025249 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:04.025272 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:04.025417 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:04.025846 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:04.025851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:04.025879 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:04.026078 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:04.026089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.027283 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.027288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:04.028809 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:04.028831 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.030036 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.030042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:04.030226 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:04.030261 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:55:04.030357 (+ 96us) client_negotiation.cc:240] Negotiation successful
0928 14:55:04.030501 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"spinlock_wait_cycles":28416}
I20260928 14:55:04.030771 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:04.025008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:33481 (local address 127.14.241.194:34935)
0928 14:55:04.025217 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:04.025224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:04.025398 (+ 174us) server_negotiation.cc:407] Connection header received
0928 14:55:04.025601 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:04.025607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:04.025685 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:04.025789 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:04.026224 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.027138 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.028968 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.029897 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.030009 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:04.030059 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:04.030400 (+ 341us) server_negotiation.cc:299] Negotiation successful
0928 14:55:04.030520 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 14:55:04.031174 1032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:04.025146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:41923 (local address 127.14.241.193:40519)
0928 14:55:04.025346 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:04.025352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:04.025861 (+ 509us) server_negotiation.cc:407] Connection header received
0928 14:55:04.026090 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:04.026096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:04.026184 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:04.026265 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:04.026807 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.027609 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.029444 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.030304 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.030448 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:04.030508 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:04.030811 (+ 303us) server_negotiation.cc:299] Negotiation successful
0928 14:55:04.030943 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 14:55:04.031431 1042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:04.024845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:40519 (local address 127.14.241.195:41923)
0928 14:55:04.025754 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:04.025770 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:04.025962 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:04.026297 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:04.026301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:04.026326 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:04.026682 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:04.026693 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.027746 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.027749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:04.029268 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:04.029282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.030468 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.030474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:04.030644 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:04.030675 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 14:55:04.030764 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 14:55:04.031025 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":2,"spinlock_wait_cycles":23296,"thread_start_us":526,"threads_started":1}
I20260928 14:55:04.031885 838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3cec26dc92941068431b4b8ddc08f22" candidate_uuid: "021cdb23d99e4b84a7515be70e160cf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ea757cb41894242b4c3cf7b04b157d9" is_pre_election: true
I20260928 14:55:04.032457 838 raft_consensus.cc:2454] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 021cdb23d99e4b84a7515be70e160cf9 in term 0.
I20260928 14:55:04.032480 706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3cec26dc92941068431b4b8ddc08f22" candidate_uuid: "021cdb23d99e4b84a7515be70e160cf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" is_pre_election: true
I20260928 14:55:04.033016 706 raft_consensus.cc:2454] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 021cdb23d99e4b84a7515be70e160cf9 in term 0.
I20260928 14:55:04.033102 905 leader_election.cc:302] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021cdb23d99e4b84a7515be70e160cf9, 6ea757cb41894242b4c3cf7b04b157d9; no voters:
I20260928 14:55:04.033483 1040 raft_consensus.cc:2781] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:55:04.033643 1040 raft_consensus.cc:482] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:55:04.033740 1040 raft_consensus.cc:3037] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:04.035930 1040 raft_consensus.cc:504] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021cdb23d99e4b84a7515be70e160cf9" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 33401 } } peers { permanent_uuid: "6ea757cb41894242b4c3cf7b04b157d9" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34935 } } peers { permanent_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40519 } }
I20260928 14:55:04.036537 1040 leader_election.cc:288] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [CANDIDATE]: Term 1 election: Requested vote from peers 6ea757cb41894242b4c3cf7b04b157d9 (127.14.241.194:34935), 64bd8c147c8249e3bd2a7977fee21ff2 (127.14.241.193:40519)
I20260928 14:55:04.036943 838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3cec26dc92941068431b4b8ddc08f22" candidate_uuid: "021cdb23d99e4b84a7515be70e160cf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ea757cb41894242b4c3cf7b04b157d9"
I20260928 14:55:04.037011 706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3cec26dc92941068431b4b8ddc08f22" candidate_uuid: "021cdb23d99e4b84a7515be70e160cf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bd8c147c8249e3bd2a7977fee21ff2"
I20260928 14:55:04.037113 838 raft_consensus.cc:3037] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:04.037220 706 raft_consensus.cc:3037] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:04.039364 838 raft_consensus.cc:2454] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 021cdb23d99e4b84a7515be70e160cf9 in term 1.
I20260928 14:55:04.039436 706 raft_consensus.cc:2454] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 021cdb23d99e4b84a7515be70e160cf9 in term 1.
I20260928 14:55:04.039739 905 leader_election.cc:302] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021cdb23d99e4b84a7515be70e160cf9, 6ea757cb41894242b4c3cf7b04b157d9; no voters:
I20260928 14:55:04.040025 1040 raft_consensus.cc:2781] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:04.040434 1040 raft_consensus.cc:686] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 1 LEADER]: Becoming Leader. State: Replica: 021cdb23d99e4b84a7515be70e160cf9, State: Running, Role: LEADER
I20260928 14:55:04.040868 1040 consensus_queue.cc:237] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021cdb23d99e4b84a7515be70e160cf9" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 33401 } } peers { permanent_uuid: "6ea757cb41894242b4c3cf7b04b157d9" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34935 } } peers { permanent_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40519 } }
I20260928 14:55:04.045423 563 catalog_manager.cc:5495] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 021cdb23d99e4b84a7515be70e160cf9 (127.14.241.195). New cstate: current_term: 1 leader_uuid: "021cdb23d99e4b84a7515be70e160cf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021cdb23d99e4b84a7515be70e160cf9" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 33401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ea757cb41894242b4c3cf7b04b157d9" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40519 } health_report { overall_health: UNKNOWN } } }
W20260928 14:55:04.079656 1016 tablet.cc:2369] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:55:04.082190 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:04.077547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.14.241.195:33401)
0928 14:55:04.077714 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:04.077718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:04.077736 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:55:04.077882 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:04.077886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:04.077970 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:04.078086 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:55:04.078368 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.079062 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.080387 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.080781 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.080872 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:04.081289 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:55:04.081786 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:55:04.081887 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:04.081940 (+ 53us) server_negotiation.cc:299] Negotiation successful
0928 14:55:04.082004 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:55:04.090451 706 raft_consensus.cc:1261] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2 [term 1 FOLLOWER]: Refusing update from remote peer 021cdb23d99e4b84a7515be70e160cf9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:04.090451 838 raft_consensus.cc:1261] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9 [term 1 FOLLOWER]: Refusing update from remote peer 021cdb23d99e4b84a7515be70e160cf9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:04.091429 1043 consensus_queue.cc:1035] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64bd8c147c8249e3bd2a7977fee21ff2" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 40519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:04.091797 1040 consensus_queue.cc:1035] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ea757cb41894242b4c3cf7b04b157d9" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:04.103526 1046 mvcc.cc:205] Tried to move back new op lower bound from 7334327517543186432 to 7334327517354983424. Current Snapshot: MvccSnapshot[applied={T|T < 7334327517543186432}]
I20260928 14:55:04.117251 562 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:04.117587 562 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:04.121709 562 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64a17e1ee96442ccbc8e0507e0226639: Sending DeleteTablet for 3 replicas of tablet c3cec26dc92941068431b4b8ddc08f22
I20260928 14:55:04.122884 818 tablet_service.cc:1514] Processing DeleteTablet for tablet c3cec26dc92941068431b4b8ddc08f22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:04 UTC) from {username='slave'} at 127.0.0.1:43334
I20260928 14:55:04.122840 949 tablet_service.cc:1514] Processing DeleteTablet for tablet c3cec26dc92941068431b4b8ddc08f22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:04 UTC) from {username='slave'} at 127.0.0.1:60998
I20260928 14:55:04.123149 686 tablet_service.cc:1514] Processing DeleteTablet for tablet c3cec26dc92941068431b4b8ddc08f22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:04 UTC) from {username='slave'} at 127.0.0.1:55292
I20260928 14:55:04.124909 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 623
I20260928 14:55:04.137598 1057 tablet_replica.cc:330] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: stopping tablet replica
I20260928 14:55:04.138270 1057 raft_consensus.cc:2229] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:55:04.138370 1058 tablet_replica.cc:330] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9: stopping tablet replica
I20260928 14:55:04.138823 1058 raft_consensus.cc:2229] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:04.139292 1057 raft_consensus.cc:2258] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:04.139381 1058 raft_consensus.cc:2258] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:04.140187 1059 tablet_replica.cc:330] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2: stopping tablet replica
I20260928 14:55:04.140637 1059 raft_consensus.cc:2229] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:04.141160 1059 raft_consensus.cc:2258] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:04.141568 1057 ts_tablet_manager.cc:1905] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:04.143151 1059 ts_tablet_manager.cc:1905] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:04.149629 1057 ts_tablet_manager.cc:1918] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:04.149829 1057 log.cc:1196] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/c3cec26dc92941068431b4b8ddc08f22
I20260928 14:55:04.150230 1059 ts_tablet_manager.cc:1918] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:04.150382 1057 ts_tablet_manager.cc:1939] T c3cec26dc92941068431b4b8ddc08f22 P 021cdb23d99e4b84a7515be70e160cf9: Deleting consensus metadata
I20260928 14:55:04.150491 1058 ts_tablet_manager.cc:1905] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:04.150535 1059 log.cc:1196] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/c3cec26dc92941068431b4b8ddc08f22
I20260928 14:55:04.151111 1059 ts_tablet_manager.cc:1939] T c3cec26dc92941068431b4b8ddc08f22 P 64bd8c147c8249e3bd2a7977fee21ff2: Deleting consensus metadata
I20260928 14:55:04.152338 548 catalog_manager.cc:4841] TS 021cdb23d99e4b84a7515be70e160cf9 (127.14.241.195:33401): tablet c3cec26dc92941068431b4b8ddc08f22 (table test-table [id=a070c1332ec3458a80799a715e535832]) successfully deleted
I20260928 14:55:04.152877 549 catalog_manager.cc:4841] TS 64bd8c147c8249e3bd2a7977fee21ff2 (127.14.241.193:40519): tablet c3cec26dc92941068431b4b8ddc08f22 (table test-table [id=a070c1332ec3458a80799a715e535832]) successfully deleted
I20260928 14:55:04.154578 1032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:04.125883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55302 (local address 127.14.241.193:40519)
0928 14:55:04.126051 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:04.126056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:04.141948 (+ 15892us) server_negotiation.cc:407] Connection header received
0928 14:55:04.142232 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:04.142239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:04.142355 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:04.142434 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:04.143102 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.144260 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.146257 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.147008 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.147115 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:04.153223 (+ 6108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:04.153231 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:04.153246 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:04.153797 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:04.153855 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:04.153877 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:04.153979 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:04.154144 (+ 165us) server_negotiation.cc:299] Negotiation successful
0928 14:55:04.154261 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:55:04.155257 747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:04.158310 1058 ts_tablet_manager.cc:1918] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:04.158551 1058 log.cc:1196] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/c3cec26dc92941068431b4b8ddc08f22
I20260928 14:55:04.159080 1058 ts_tablet_manager.cc:1939] T c3cec26dc92941068431b4b8ddc08f22 P 6ea757cb41894242b4c3cf7b04b157d9: Deleting consensus metadata
I20260928 14:55:04.160984 551 catalog_manager.cc:4841] TS 6ea757cb41894242b4c3cf7b04b157d9 (127.14.241.194:34935): tablet c3cec26dc92941068431b4b8ddc08f22 (table test-table [id=a070c1332ec3458a80799a715e535832]) successfully deleted
I20260928 14:55:04.244518 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 623
I20260928 14:55:04.267309 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 756
I20260928 14:55:04.274535 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:04.268074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43336 (local address 127.14.241.194:34935)
0928 14:55:04.268240 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:04.268245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:04.268262 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:55:04.268483 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:04.268487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:04.268572 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:04.268655 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:04.269105 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.269919 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.271739 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.272120 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.272204 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:04.273258 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:04.273266 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:04.273280 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:04.273861 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:04.273908 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:04.273928 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:04.274009 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:04.274142 (+ 133us) server_negotiation.cc:299] Negotiation successful
0928 14:55:04.274228 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:55:04.275177 879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:04.370203 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 756
I20260928 14:55:04.391095 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 887
I20260928 14:55:04.398126 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:04.391946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.14.241.195:33401)
0928 14:55:04.392125 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:04.392131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:04.392152 (+ 21us) server_negotiation.cc:407] Connection header received
0928 14:55:04.392266 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:04.392272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:04.392414 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:04.392508 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:04.392961 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.393769 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.395107 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:04.395549 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:04.395670 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:04.396752 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:04.396761 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:04.396775 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:04.397408 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:04.397459 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:04.397482 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:04.397636 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:04.397693 (+ 57us) server_negotiation.cc:299] Negotiation successful
0928 14:55:04.397792 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:55:04.398805 1010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:04.518424 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 887
I20260928 14:55:04.535208 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 533
I20260928 14:55:04.536365 594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:04.646155 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 533
2026-09-28T14:55:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2779 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:04 dist-test-slave-vk40 krb5kdc[1067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:04 dist-test-slave-vk40 krb5kdc[1067](info): set up 2 sockets
Sep 28 14:55:04 dist-test-slave-vk40 krb5kdc[1067](info): commencing operation
krb5kdc: starting...
W20260928 14:55:05.659042 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:05 dist-test-slave-vk40 krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:55:05.899588 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41075
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:41075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:06.224062 1080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:06.224434 1080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:06.224574 1080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:06.232168 1080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:06.232338 1080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:06.232421 1080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:06.232499 1080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:06.232558 1080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:06.242936 1080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:41075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:06.244321 1080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:06.245690 1080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:06.257331 1086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:06.257429 1088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:06.257699 1085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:06.258137 1080 server_base.cc:1034] running on GCE node
I20260928 14:55:06.258976 1080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:06.260598 1080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:06.261814 1080 hybrid_clock.cc:648] HybridClock initialized: now 1790607306261749 us; error 50 us; skew 500 ppm
I20260928 14:55:06.262161 1080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:06 dist-test-slave-vk40 krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607306, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:06.265239 1080 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:55:06.266731 1080 webserver.cc:460] Webserver started at http://127.14.241.254:35935/ using document root <none> and password file <none>
I20260928 14:55:06.267385 1080 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:06.267484 1080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:06.267751 1080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:06.270790 1080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "fcb88158dd1143ba965b17c59179e3ec"
format_stamp: "Formatted at 2026-09-28 14:55:06 on dist-test-slave-vk40"
server_key: "03ba8108409358bdf35666caa229f5e6"
server_key_iv: "5a8a0c6076d4e01fc7e0c2c4c138f5b4"
server_key_version: "encryptionkey@0"
I20260928 14:55:06.271577 1080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "fcb88158dd1143ba965b17c59179e3ec"
format_stamp: "Formatted at 2026-09-28 14:55:06 on dist-test-slave-vk40"
server_key: "03ba8108409358bdf35666caa229f5e6"
server_key_iv: "5a8a0c6076d4e01fc7e0c2c4c138f5b4"
server_key_version: "encryptionkey@0"
I20260928 14:55:06.276983 1080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 14:55:06.280697 1095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:06.282148 1080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:55:06.282303 1080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "fcb88158dd1143ba965b17c59179e3ec"
format_stamp: "Formatted at 2026-09-28 14:55:06 on dist-test-slave-vk40"
server_key: "03ba8108409358bdf35666caa229f5e6"
server_key_iv: "5a8a0c6076d4e01fc7e0c2c4c138f5b4"
server_key_version: "encryptionkey@0"
I20260928 14:55:06.282483 1080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:06.300690 1080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:06.305079 1080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:06.305328 1080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:06.329089 1080 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:41075
I20260928 14:55:06.329097 1146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:41075 every 8 connection(s)
I20260928 14:55:06.331017 1080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:06.334177 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 1080
I20260928 14:55:06.334348 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:06.334781 15303 external_mini_cluster.cc:1428] Setting key 2990ab226ab97297d97c4ce08803dfcc
I20260928 14:55:06.335300 1147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:06.349720 1147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec: Bootstrap starting.
Sep 28 14:55:06 dist-test-slave-vk40 krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:06.353048 1147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:06.354188 1147 log.cc:824] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:06.357190 1147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec: No bootstrap required, opened a new log
I20260928 14:55:06.358850 1150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:06.336609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40850 (local address 127.14.241.254:41075)
0928 14:55:06.337396 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:06.337409 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:06.337449 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:55:06.340012 (+ 2563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:06.340037 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:06.340542 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:06.340913 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:06.342754 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:06.344061 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:06.346000 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:06.346794 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:06.346949 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:06.351584 (+ 4635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:06.351595 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:06.351614 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:06.351660 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:06.354680 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:06.355441 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:06.355449 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:06.355456 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:06.355644 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:06.356020 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:06.356026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:06.356028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:06.356660 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:06.356965 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:06.357395 (+ 430us) server_negotiation.cc:299] Negotiation successful
0928 14:55:06.357843 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":357,"threads_started":1}
I20260928 14:55:06.365298 1147 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb88158dd1143ba965b17c59179e3ec" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41075 } }
I20260928 14:55:06.365679 1147 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:06.365773 1147 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb88158dd1143ba965b17c59179e3ec, State: Initialized, Role: FOLLOWER
I20260928 14:55:06.366307 1147 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb88158dd1143ba965b17c59179e3ec" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41075 } }
I20260928 14:55:06.366523 1147 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:06.366657 1147 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:06.366806 1147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:06.369010 1147 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb88158dd1143ba965b17c59179e3ec" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41075 } }
I20260928 14:55:06.369509 1147 leader_election.cc:302] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcb88158dd1143ba965b17c59179e3ec; no voters:
I20260928 14:55:06.370123 1147 leader_election.cc:288] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:06.370271 1152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:06.371165 1152 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [term 1 LEADER]: Becoming Leader. State: Replica: fcb88158dd1143ba965b17c59179e3ec, State: Running, Role: LEADER
I20260928 14:55:06.371696 1152 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb88158dd1143ba965b17c59179e3ec" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41075 } }
I20260928 14:55:06.372373 1147 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:06.374541 1154 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcb88158dd1143ba965b17c59179e3ec. Latest consensus state: current_term: 1 leader_uuid: "fcb88158dd1143ba965b17c59179e3ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb88158dd1143ba965b17c59179e3ec" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41075 } } }
I20260928 14:55:06.374461 1153 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcb88158dd1143ba965b17c59179e3ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb88158dd1143ba965b17c59179e3ec" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41075 } } }
I20260928 14:55:06.374890 1154 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:06.374890 1153 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:06.375739 1160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:06.381354 1160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:06.390519 1160 catalog_manager.cc:1347] Generated new cluster ID: 56734cb1267a4f6693550785a0bcac3a
I20260928 14:55:06.390637 1160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:06.413014 1160 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:06.415023 1160 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:06.425745 1160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec: Generated new TSK 0
I20260928 14:55:06.426954 1160 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:55:06.496482 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41075
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:06.829244 1173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:06.829551 1173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:06.829699 1173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:06.837033 1173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:06.837170 1173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:06.837333 1173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:55:06.847324 1173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:06.848707 1173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:06.850098 1173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:06.863214 1178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:06.863113 1181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:06.863142 1179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:06.863922 1173 server_base.cc:1034] running on GCE node
I20260928 14:55:06.864557 1173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:06.865465 1173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:06.866672 1173 hybrid_clock.cc:648] HybridClock initialized: now 1790607306866608 us; error 53 us; skew 500 ppm
I20260928 14:55:06.866978 1173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:06 dist-test-slave-vk40 krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607306, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:06.870047 1173 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:55:06.871487 1173 webserver.cc:460] Webserver started at http://127.14.241.193:40375/ using document root <none> and password file <none>
I20260928 14:55:06.872228 1173 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:06.872381 1173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:06.872674 1173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:06.875792 1173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "fc4dfbf157014a1b81bfc51989734364"
format_stamp: "Formatted at 2026-09-28 14:55:06 on dist-test-slave-vk40"
server_key: "14ef112ec90360800e42f510e0490f15"
server_key_iv: "dee18fb8f72c5080194f02a81adaf5f7"
server_key_version: "encryptionkey@0"
I20260928 14:55:06.876614 1173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "fc4dfbf157014a1b81bfc51989734364"
format_stamp: "Formatted at 2026-09-28 14:55:06 on dist-test-slave-vk40"
server_key: "14ef112ec90360800e42f510e0490f15"
server_key_iv: "dee18fb8f72c5080194f02a81adaf5f7"
server_key_version: "encryptionkey@0"
I20260928 14:55:06.882064 1173 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:55:06.885728 1188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:06.887247 1173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:55:06.887403 1173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "fc4dfbf157014a1b81bfc51989734364"
format_stamp: "Formatted at 2026-09-28 14:55:06 on dist-test-slave-vk40"
server_key: "14ef112ec90360800e42f510e0490f15"
server_key_iv: "dee18fb8f72c5080194f02a81adaf5f7"
server_key_version: "encryptionkey@0"
I20260928 14:55:06.887590 1173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:06.905380 1173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:06.909468 1173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:06.909744 1173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:06.910727 1173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:06.912134 1173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:06.912207 1173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:06.912356 1173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:06.912420 1173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:06.943259 1173 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:41519
I20260928 14:55:06.943276 1303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:41519 every 8 connection(s)
I20260928 14:55:06.944630 1173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:55:06.953506 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 1173
I20260928 14:55:06.953666 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:55:06.954131 15303 external_mini_cluster.cc:1428] Setting key 3ec53b04e3294aaa2468df3aca63253f
Sep 28 14:55:06 dist-test-slave-vk40 krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607306, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:06.965309 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:06.947604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:46753 (local address 127.14.241.254:41075)
0928 14:55:06.948734 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:06.948742 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:06.949437 (+ 695us) server_negotiation.cc:407] Connection header received
0928 14:55:06.951344 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:06.951349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:06.951429 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:06.951531 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:06.953142 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:06.953948 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:06.955773 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:06.956188 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:06.956331 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:06.959777 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:06.959785 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:06.959789 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:06.959824 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:06.962168 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:06.962944 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:06.962948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:06.962950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:06.963075 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:06.963505 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:06.963508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:06.963511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:06.963788 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:06.963933 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:06.964699 (+ 766us) server_negotiation.cc:299] Negotiation successful
0928 14:55:06.964955 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":991,"thread_start_us":819,"threads_started":1}
I20260928 14:55:06.966254 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:06.948315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41075 (local address 127.14.241.193:46753)
0928 14:55:06.949245 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:06.949301 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:06.951061 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:06.951743 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:06.951752 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:06.952352 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:06.952964 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:06.952984 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:06.954123 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:06.954129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:06.955620 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:06.955634 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:06.956459 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:06.956466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:06.956709 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:06.957379 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:06.957406 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:06.959535 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:06.962310 (+ 2775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:06.962319 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:06.962337 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:06.962805 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:06.963179 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:06.963184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:06.963188 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:06.963389 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:06.963912 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:06.963920 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:06.964303 (+ 383us) client_negotiation.cc:769] Sending connection context
0928 14:55:06.964573 (+ 270us) client_negotiation.cc:240] Negotiation successful
0928 14:55:06.965111 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":373,"threads_started":1}
I20260928 14:55:06.968000 1304 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41075
I20260928 14:55:06.968320 1304 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:06.969054 1304 heartbeater.cc:511] Master 127.14.241.254:41075 requested a full tablet report, sending...
I20260928 14:55:06.971367 1112 ts_manager.cc:194] Registered new tserver with Master: fc4dfbf157014a1b81bfc51989734364 (127.14.241.193:41519)
I20260928 14:55:06.973450 1112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:46753
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:55:07.008114 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41075
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:07.344195 1310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:07.344497 1310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:07.344626 1310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:07.351851 1310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:07.351979 1310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:07.352142 1310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:55:07.361990 1310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:07.363233 1310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:07.364576 1310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:07.377386 1316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:07.377389 1315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:07.377449 1318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:07.378144 1310 server_base.cc:1034] running on GCE node
I20260928 14:55:07.378786 1310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:07.379717 1310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:07.380905 1310 hybrid_clock.cc:648] HybridClock initialized: now 1790607307380843 us; error 54 us; skew 500 ppm
I20260928 14:55:07.381204 1310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:07 dist-test-slave-vk40 krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607307, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:07.384219 1310 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:55:07.385740 1310 webserver.cc:460] Webserver started at http://127.14.241.194:36895/ using document root <none> and password file <none>
I20260928 14:55:07.386677 1310 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:07.386797 1310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:07.387111 1310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:07.390019 1310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "dc50800f27034f8285e0f86a6633975d"
format_stamp: "Formatted at 2026-09-28 14:55:07 on dist-test-slave-vk40"
server_key: "e360a9db28c1fa4d3b0cc30a99c43c74"
server_key_iv: "009d4fbf53d3c300df00d6c61e0b83ef"
server_key_version: "encryptionkey@0"
I20260928 14:55:07.390723 1310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "dc50800f27034f8285e0f86a6633975d"
format_stamp: "Formatted at 2026-09-28 14:55:07 on dist-test-slave-vk40"
server_key: "e360a9db28c1fa4d3b0cc30a99c43c74"
server_key_iv: "009d4fbf53d3c300df00d6c61e0b83ef"
server_key_version: "encryptionkey@0"
I20260928 14:55:07.395696 1310 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260928 14:55:07.399070 1325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:07.400434 1310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260928 14:55:07.400579 1310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "dc50800f27034f8285e0f86a6633975d"
format_stamp: "Formatted at 2026-09-28 14:55:07 on dist-test-slave-vk40"
server_key: "e360a9db28c1fa4d3b0cc30a99c43c74"
server_key_iv: "009d4fbf53d3c300df00d6c61e0b83ef"
server_key_version: "encryptionkey@0"
I20260928 14:55:07.400756 1310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:07.416957 1310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:07.420809 1310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:07.421078 1310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:07.422020 1310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:07.423353 1310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:07.423445 1310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:07.423564 1310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:07.423616 1310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:07.453094 1310 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:34963
I20260928 14:55:07.453130 1438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:34963 every 8 connection(s)
I20260928 14:55:07.454382 1310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:55:07.460619 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 1310
I20260928 14:55:07.460781 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:55:07.461187 15303 external_mini_cluster.cc:1428] Setting key c94a83f102ebd0671126e920b3ee165e
Sep 28 14:55:07 dist-test-slave-vk40 krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607307, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:07.473501 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:07.456766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:36495 (local address 127.14.241.254:41075)
0928 14:55:07.456940 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:07.456945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:07.458103 (+ 1158us) server_negotiation.cc:407] Connection header received
0928 14:55:07.459943 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:07.459949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:07.460033 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:07.460123 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:07.461794 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:07.462515 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:07.464065 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:07.464539 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:07.464652 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:07.467964 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:07.467969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:07.467973 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:07.468006 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:07.470410 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:07.471114 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:07.471118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:07.471120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:07.471253 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:07.471660 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:07.471663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:07.471665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:07.471932 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:07.472067 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:07.472843 (+ 776us) server_negotiation.cc:299] Negotiation successful
0928 14:55:07.473086 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:55:07.474426 1441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:07.457219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41075 (local address 127.14.241.194:36495)
0928 14:55:07.457932 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:07.457977 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:07.459658 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:07.460414 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:07.460423 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:07.461009 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:07.461619 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:07.461637 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:07.462664 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:07.462668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:07.463923 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:07.463935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:07.464750 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:07.464757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:07.464991 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:07.465611 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:07.465633 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:07.467736 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:07.470545 (+ 2809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:07.470554 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:07.470573 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:07.470998 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:07.471356 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:07.471361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:07.471365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:07.471555 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:07.472054 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:07.472062 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:07.472469 (+ 407us) client_negotiation.cc:769] Sending connection context
0928 14:55:07.472706 (+ 237us) client_negotiation.cc:240] Negotiation successful
0928 14:55:07.473280 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":260,"threads_started":1}
I20260928 14:55:07.475841 1439 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41075
I20260928 14:55:07.476161 1439 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:07.477034 1439 heartbeater.cc:511] Master 127.14.241.254:41075 requested a full tablet report, sending...
I20260928 14:55:07.478610 1112 ts_manager.cc:194] Registered new tserver with Master: dc50800f27034f8285e0f86a6633975d (127.14.241.194:34963)
I20260928 14:55:07.479406 1112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:36495
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:55:07.513022 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41075
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:34581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:07.833429 1444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:07.833716 1444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:07.833810 1444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:07.841379 1444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:07.841501 1444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:07.841617 1444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:55:07.851358 1444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:34581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:41075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:07.852584 1444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:07.853826 1444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:07.866604 1450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:07.866667 1449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:07.866698 1452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:07.867290 1444 server_base.cc:1034] running on GCE node
I20260928 14:55:07.867883 1444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:07.868854 1444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:07.870057 1444 hybrid_clock.cc:648] HybridClock initialized: now 1790607307869999 us; error 48 us; skew 500 ppm
I20260928 14:55:07.870353 1444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:07 dist-test-slave-vk40 krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607307, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:07.878432 1444 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:55:07.879941 1444 webserver.cc:460] Webserver started at http://127.14.241.195:38715/ using document root <none> and password file <none>
I20260928 14:55:07.880714 1444 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:07.880831 1444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:07.881139 1444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:07.884147 1444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "f50385b4520d4120a7750ad080a72f3a"
format_stamp: "Formatted at 2026-09-28 14:55:07 on dist-test-slave-vk40"
server_key: "e5a3c466628ded1bb0775e95364888d4"
server_key_iv: "1c27c329e8da133550a95d7ad3ddb5c4"
server_key_version: "encryptionkey@0"
I20260928 14:55:07.884990 1444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "f50385b4520d4120a7750ad080a72f3a"
format_stamp: "Formatted at 2026-09-28 14:55:07 on dist-test-slave-vk40"
server_key: "e5a3c466628ded1bb0775e95364888d4"
server_key_iv: "1c27c329e8da133550a95d7ad3ddb5c4"
server_key_version: "encryptionkey@0"
I20260928 14:55:07.890280 1444 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:07.893798 1460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:07.895138 1444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:55:07.895278 1444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "f50385b4520d4120a7750ad080a72f3a"
format_stamp: "Formatted at 2026-09-28 14:55:07 on dist-test-slave-vk40"
server_key: "e5a3c466628ded1bb0775e95364888d4"
server_key_iv: "1c27c329e8da133550a95d7ad3ddb5c4"
server_key_version: "encryptionkey@0"
I20260928 14:55:07.895470 1444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:07.909628 1444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:07.913441 1444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:07.913707 1444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:07.914624 1444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:07.915940 1444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:07.916011 1444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:07.916131 1444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:07.916184 1444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:07.946229 1444 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:42909
I20260928 14:55:07.946271 1572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:42909 every 8 connection(s)
I20260928 14:55:07.947553 1444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:55:07.956248 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 1444
I20260928 14:55:07.956457 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:55:07.956959 15303 external_mini_cluster.cc:1428] Setting key cf89ee4c48a7c7319a5d74bf1c62a2fe
Sep 28 14:55:07 dist-test-slave-vk40 krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607307, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:07.967597 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:07.950477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:57009 (local address 127.14.241.254:41075)
0928 14:55:07.950696 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:07.950702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:07.951932 (+ 1230us) server_negotiation.cc:407] Connection header received
0928 14:55:07.953945 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:07.953951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:07.954068 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:07.954158 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:07.955638 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:07.956691 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:07.958409 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:07.958920 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:07.959012 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:07.962513 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:07.962518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:07.962521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:07.962557 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:07.964883 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:07.965576 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:07.965580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:07.965583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:07.965733 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:07.966081 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:07.966084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:07.966087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:07.966375 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:07.966496 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:07.967170 (+ 674us) server_negotiation.cc:299] Negotiation successful
0928 14:55:07.967352 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260928 14:55:07.968670 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:07.950911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:41075 (local address 127.14.241.195:57009)
0928 14:55:07.951775 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:07.951823 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:07.953663 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:07.954406 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:07.954414 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:07.954915 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:07.955473 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:07.955490 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:07.956849 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:07.956853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:07.958269 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:07.958283 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:07.959140 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:07.959148 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:07.959451 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:07.960141 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:07.960164 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:07.962306 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:07.965008 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:07.965016 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:07.965030 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:07.965452 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:07.965823 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:07.965826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:07.965828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:07.966004 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:07.966466 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:07.966472 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:07.966823 (+ 351us) client_negotiation.cc:769] Sending connection context
0928 14:55:07.967050 (+ 227us) client_negotiation.cc:240] Negotiation successful
0928 14:55:07.967454 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":339,"threads_started":1}
I20260928 14:55:07.970324 1573 heartbeater.cc:347] Connected to a master server at 127.14.241.254:41075
I20260928 14:55:07.970649 1573 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:07.971462 1573 heartbeater.cc:511] Master 127.14.241.254:41075 requested a full tablet report, sending...
I20260928 14:55:07.972954 1112 ts_manager.cc:194] Registered new tserver with Master: f50385b4520d4120a7750ad080a72f3a (127.14.241.195:42909)
I20260928 14:55:07.973816 1112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:57009
I20260928 14:55:07.976722 1304 heartbeater.cc:503] Master 127.14.241.254:41075 was elected leader, sending a full tablet report...
I20260928 14:55:07.984267 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:55:08.000985 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:07.988382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.14.241.254:41075)
0928 14:55:07.988538 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:07.988543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:07.988952 (+ 409us) server_negotiation.cc:407] Connection header received
0928 14:55:07.989094 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:07.989097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:07.989163 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:07.989252 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:07.990740 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:07.991506 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:07.992888 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:07.993286 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:07.993390 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:07.995872 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:07.995877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:07.995880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:07.995914 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:07.998403 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:07.999132 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:07.999136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:07.999139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:07.999274 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:07.999689 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:07.999694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:07.999697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:08.000023 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:08.000252 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:08.000598 (+ 346us) server_negotiation.cc:299] Negotiation successful
0928 14:55:08.000748 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 14:55:08.004673 1112 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:55:08.011425 1112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:55:08.046247 1585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.024060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:34963 (local address 127.0.0.1:43454)
0928 14:55:08.036892 (+ 12832us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:08.036911 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:08.037034 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:08.037794 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:08.037801 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:08.037839 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:08.038222 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:08.038237 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.040347 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.040352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:08.042987 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:08.043006 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.045225 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.045231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:08.045461 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.045519 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 14:55:08.045644 (+ 125us) client_negotiation.cc:240] Negotiation successful
0928 14:55:08.045853 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12711,"thread_start_us":12510,"threads_started":1}
I20260928 14:55:08.046662 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.024252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43454 (local address 127.14.241.194:34963)
0928 14:55:08.036527 (+ 12275us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:08.036538 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:08.037160 (+ 622us) server_negotiation.cc:407] Connection header received
0928 14:55:08.037268 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:08.037275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:08.037543 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:08.038428 (+ 885us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:08.038655 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.040163 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.043259 (+ 3096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.044737 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.044892 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.045080 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:08.046176 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0928 14:55:08.046355 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12121,"thread_start_us":11996,"threads_started":1}
I20260928 14:55:08.047161 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.023522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:42909 (local address 127.0.0.1:44226)
0928 14:55:08.035540 (+ 12018us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:08.035617 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:08.035873 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:08.039355 (+ 3482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:08.039361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:08.039386 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:08.039793 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:08.039805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.042023 (+ 2218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.042029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:08.044109 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:08.044124 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.045922 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.045927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:08.046088 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.046115 (+ 27us) client_negotiation.cc:769] Sending connection context
0928 14:55:08.046204 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 14:55:08.046593 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11648,"mutex_wait_us":20,"thread_start_us":13105,"threads_started":1}
I20260928 14:55:08.047290 1586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.023768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44226 (local address 127.14.241.195:42909)
0928 14:55:08.038525 (+ 14757us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:08.038533 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:08.038566 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:55:08.038662 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:08.038670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:08.038899 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:08.039084 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:08.040002 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.041843 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.044275 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.045698 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.045878 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.046051 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:08.046808 (+ 757us) server_negotiation.cc:299] Negotiation successful
0928 14:55:08.046969 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14589,"thread_start_us":14443,"threads_started":1}
I20260928 14:55:08.048538 1590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.038577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:41519 (local address 127.0.0.1:51416)
0928 14:55:08.041013 (+ 2436us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:08.041035 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:08.041159 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:08.041996 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:08.042000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:08.042026 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:08.042419 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:08.042431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.043832 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.043836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:08.045798 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:08.045812 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.047752 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.047758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:08.047924 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.047954 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 14:55:08.048044 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 14:55:08.048165 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2291,"mutex_wait_us":3,"thread_start_us":1889,"threads_started":1}
I20260928 14:55:08.049387 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.037938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.14.241.193:41519)
0928 14:55:08.040419 (+ 2481us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:08.040426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:08.041220 (+ 794us) server_negotiation.cc:407] Connection header received
0928 14:55:08.041321 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:08.041328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:08.041562 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:08.041750 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:08.042581 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.043702 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.045985 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.047391 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.047579 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.047754 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:08.048106 (+ 352us) server_negotiation.cc:299] Negotiation successful
0928 14:55:08.048273 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2291,"thread_start_us":2141,"threads_started":1}
I20260928 14:55:08.050922 1374 tablet_service.cc:1467] Processing CreateTablet for tablet 26bb48b5134f43a6ad41d53b051e647c (DEFAULT_TABLE table=test-table [id=967394c9f4a2465eb3927c5e4035a791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:08.051201 1508 tablet_service.cc:1467] Processing CreateTablet for tablet 26bb48b5134f43a6ad41d53b051e647c (DEFAULT_TABLE table=test-table [id=967394c9f4a2465eb3927c5e4035a791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:08.052572 1508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26bb48b5134f43a6ad41d53b051e647c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:08.052580 1374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26bb48b5134f43a6ad41d53b051e647c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:08.052435 1238 tablet_service.cc:1467] Processing CreateTablet for tablet 26bb48b5134f43a6ad41d53b051e647c (DEFAULT_TABLE table=test-table [id=967394c9f4a2465eb3927c5e4035a791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:08.053573 1238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26bb48b5134f43a6ad41d53b051e647c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:08.065719 1591 tablet_bootstrap.cc:492] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: Bootstrap starting.
I20260928 14:55:08.069607 1591 tablet_bootstrap.cc:654] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:08.070613 1591 log.cc:824] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:08.073484 1591 tablet_bootstrap.cc:492] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: No bootstrap required, opened a new log
I20260928 14:55:08.073737 1591 ts_tablet_manager.cc:1397] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260928 14:55:08.076409 1592 tablet_bootstrap.cc:492] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: Bootstrap starting.
I20260928 14:55:08.077337 1593 tablet_bootstrap.cc:492] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: Bootstrap starting.
I20260928 14:55:08.079886 1592 tablet_bootstrap.cc:654] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:08.080917 1593 tablet_bootstrap.cc:654] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:08.080969 1592 log.cc:824] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:08.082118 1593 log.cc:824] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:08.083899 1592 tablet_bootstrap.cc:492] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: No bootstrap required, opened a new log
I20260928 14:55:08.084169 1592 ts_tablet_manager.cc:1397] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.004s
I20260928 14:55:08.084116 1591 raft_consensus.cc:348] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50385b4520d4120a7750ad080a72f3a" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42909 } } peers { permanent_uuid: "fc4dfbf157014a1b81bfc51989734364" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 41519 } } peers { permanent_uuid: "dc50800f27034f8285e0f86a6633975d" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34963 } }
I20260928 14:55:08.084612 1591 raft_consensus.cc:374] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:08.084718 1591 raft_consensus.cc:729] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50385b4520d4120a7750ad080a72f3a, State: Initialized, Role: FOLLOWER
I20260928 14:55:08.085276 1593 tablet_bootstrap.cc:492] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: No bootstrap required, opened a new log
I20260928 14:55:08.085551 1593 ts_tablet_manager.cc:1397] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260928 14:55:08.085390 1591 consensus_queue.cc:260] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50385b4520d4120a7750ad080a72f3a" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42909 } } peers { permanent_uuid: "fc4dfbf157014a1b81bfc51989734364" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 41519 } } peers { permanent_uuid: "dc50800f27034f8285e0f86a6633975d" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34963 } }
I20260928 14:55:08.086529 1573 heartbeater.cc:503] Master 127.14.241.254:41075 was elected leader, sending a full tablet report...
I20260928 14:55:08.087141 1591 ts_tablet_manager.cc:1428] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260928 14:55:08.093631 1592 raft_consensus.cc:348] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50385b4520d4120a7750ad080a72f3a" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42909 } } peers { permanent_uuid: "fc4dfbf157014a1b81bfc51989734364" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 41519 } } peers { permanent_uuid: "dc50800f27034f8285e0f86a6633975d" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34963 } }
I20260928 14:55:08.094002 1592 raft_consensus.cc:374] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:08.094092 1592 raft_consensus.cc:729] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4dfbf157014a1b81bfc51989734364, State: Initialized, Role: FOLLOWER
I20260928 14:55:08.094344 1593 raft_consensus.cc:348] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50385b4520d4120a7750ad080a72f3a" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42909 } } peers { permanent_uuid: "fc4dfbf157014a1b81bfc51989734364" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 41519 } } peers { permanent_uuid: "dc50800f27034f8285e0f86a6633975d" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34963 } }
I20260928 14:55:08.094717 1593 raft_consensus.cc:374] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:08.094617 1592 consensus_queue.cc:260] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50385b4520d4120a7750ad080a72f3a" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42909 } } peers { permanent_uuid: "fc4dfbf157014a1b81bfc51989734364" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 41519 } } peers { permanent_uuid: "dc50800f27034f8285e0f86a6633975d" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34963 } }
I20260928 14:55:08.094798 1593 raft_consensus.cc:729] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc50800f27034f8285e0f86a6633975d, State: Initialized, Role: FOLLOWER
I20260928 14:55:08.095310 1593 consensus_queue.cc:260] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50385b4520d4120a7750ad080a72f3a" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42909 } } peers { permanent_uuid: "fc4dfbf157014a1b81bfc51989734364" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 41519 } } peers { permanent_uuid: "dc50800f27034f8285e0f86a6633975d" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34963 } }
I20260928 14:55:08.096473 1592 ts_tablet_manager.cc:1428] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: Time spent starting tablet: real 0.012s user 0.003s sys 0.011s
I20260928 14:55:08.096810 1593 ts_tablet_manager.cc:1428] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260928 14:55:08.096943 1439 heartbeater.cc:503] Master 127.14.241.254:41075 was elected leader, sending a full tablet report...
I20260928 14:55:08.190960 1599 raft_consensus.cc:482] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:55:08.191321 1599 raft_consensus.cc:504] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50385b4520d4120a7750ad080a72f3a" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42909 } } peers { permanent_uuid: "fc4dfbf157014a1b81bfc51989734364" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 41519 } } peers { permanent_uuid: "dc50800f27034f8285e0f86a6633975d" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34963 } }
I20260928 14:55:08.193346 1599 leader_election.cc:288] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f50385b4520d4120a7750ad080a72f3a (127.14.241.195:42909), fc4dfbf157014a1b81bfc51989734364 (127.14.241.193:41519)
W20260928 14:55:08.196851 1305 tablet.cc:2369] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:55:08.199087 1574 tablet.cc:2369] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:55:08.200516 1586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.193763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:56885 (local address 127.14.241.195:42909)
0928 14:55:08.193976 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:08.193984 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:08.194440 (+ 456us) server_negotiation.cc:407] Connection header received
0928 14:55:08.194631 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:08.194638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:08.194719 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:08.194813 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:08.195409 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.196378 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.198355 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.199538 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.199685 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.199764 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:08.200105 (+ 341us) server_negotiation.cc:299] Negotiation successful
0928 14:55:08.200247 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 14:55:08.200538 1600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.193705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:42909 (local address 127.14.241.194:56885)
0928 14:55:08.194328 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:08.194344 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:08.194495 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:08.194858 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:08.194863 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:08.194893 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:08.195254 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:08.195267 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.196498 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.196504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:08.198184 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:08.198197 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.199744 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.199751 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:08.199937 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.199963 (+ 26us) client_negotiation.cc:769] Sending connection context
0928 14:55:08.200063 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 14:55:08.200171 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":336,"threads_started":1}
I20260928 14:55:08.201222 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.194557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:58085 (local address 127.14.241.193:41519)
0928 14:55:08.194713 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:08.194718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:08.195121 (+ 403us) server_negotiation.cc:407] Connection header received
0928 14:55:08.195253 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:08.195259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:08.195339 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:08.195423 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:08.195969 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.196877 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.199075 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.200318 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.200488 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.200559 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:08.200909 (+ 350us) server_negotiation.cc:299] Negotiation successful
0928 14:55:08.201049 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:55:08.201709 1528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26bb48b5134f43a6ad41d53b051e647c" candidate_uuid: "dc50800f27034f8285e0f86a6633975d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f50385b4520d4120a7750ad080a72f3a" is_pre_election: true
I20260928 14:55:08.202196 1528 raft_consensus.cc:2454] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc50800f27034f8285e0f86a6633975d in term 0.
I20260928 14:55:08.202589 1601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.194405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:41519 (local address 127.14.241.194:58085)
0928 14:55:08.195019 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:08.195033 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:08.195133 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:08.195454 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:08.195459 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:08.195485 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:08.195811 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:08.195822 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.197068 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.197072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:08.198906 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:08.198923 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.200551 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.200559 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:08.200746 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.200776 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 14:55:08.200884 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 14:55:08.201014 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":304,"threads_started":1}
I20260928 14:55:08.202896 1329 leader_election.cc:302] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc50800f27034f8285e0f86a6633975d, f50385b4520d4120a7750ad080a72f3a; no voters:
I20260928 14:55:08.203282 1599 raft_consensus.cc:2781] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:55:08.203439 1599 raft_consensus.cc:482] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:55:08.203540 1599 raft_consensus.cc:3037] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:08.204046 1258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26bb48b5134f43a6ad41d53b051e647c" candidate_uuid: "dc50800f27034f8285e0f86a6633975d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4dfbf157014a1b81bfc51989734364" is_pre_election: true
I20260928 14:55:08.204586 1258 raft_consensus.cc:2454] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc50800f27034f8285e0f86a6633975d in term 0.
W20260928 14:55:08.206265 1440 tablet.cc:2369] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:55:08.206282 1599 raft_consensus.cc:504] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50385b4520d4120a7750ad080a72f3a" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42909 } } peers { permanent_uuid: "fc4dfbf157014a1b81bfc51989734364" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 41519 } } peers { permanent_uuid: "dc50800f27034f8285e0f86a6633975d" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34963 } }
I20260928 14:55:08.207151 1599 leader_election.cc:288] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [CANDIDATE]: Term 1 election: Requested vote from peers f50385b4520d4120a7750ad080a72f3a (127.14.241.195:42909), fc4dfbf157014a1b81bfc51989734364 (127.14.241.193:41519)
I20260928 14:55:08.207628 1528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26bb48b5134f43a6ad41d53b051e647c" candidate_uuid: "dc50800f27034f8285e0f86a6633975d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f50385b4520d4120a7750ad080a72f3a"
I20260928 14:55:08.207651 1258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26bb48b5134f43a6ad41d53b051e647c" candidate_uuid: "dc50800f27034f8285e0f86a6633975d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4dfbf157014a1b81bfc51989734364"
I20260928 14:55:08.207831 1528 raft_consensus.cc:3037] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:08.207840 1258 raft_consensus.cc:3037] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:08.210052 1258 raft_consensus.cc:2454] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc50800f27034f8285e0f86a6633975d in term 1.
I20260928 14:55:08.210060 1528 raft_consensus.cc:2454] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc50800f27034f8285e0f86a6633975d in term 1.
I20260928 14:55:08.210472 1329 leader_election.cc:302] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc50800f27034f8285e0f86a6633975d, f50385b4520d4120a7750ad080a72f3a; no voters:
I20260928 14:55:08.210767 1599 raft_consensus.cc:2781] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:08.211191 1599 raft_consensus.cc:686] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 1 LEADER]: Becoming Leader. State: Replica: dc50800f27034f8285e0f86a6633975d, State: Running, Role: LEADER
I20260928 14:55:08.211733 1599 consensus_queue.cc:237] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50385b4520d4120a7750ad080a72f3a" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42909 } } peers { permanent_uuid: "fc4dfbf157014a1b81bfc51989734364" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 41519 } } peers { permanent_uuid: "dc50800f27034f8285e0f86a6633975d" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34963 } }
I20260928 14:55:08.216252 1112 catalog_manager.cc:5495] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d reported cstate change: term changed from 0 to 1, leader changed from <none> to dc50800f27034f8285e0f86a6633975d (127.14.241.194). New cstate: current_term: 1 leader_uuid: "dc50800f27034f8285e0f86a6633975d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50385b4520d4120a7750ad080a72f3a" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc4dfbf157014a1b81bfc51989734364" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc50800f27034f8285e0f86a6633975d" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 34963 } health_report { overall_health: HEALTHY } } }
I20260928 14:55:08.241163 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.235878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43468 (local address 127.14.241.194:34963)
0928 14:55:08.236071 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:08.236076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:08.236099 (+ 23us) server_negotiation.cc:407] Connection header received
0928 14:55:08.236347 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:08.236351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:08.236428 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:08.236554 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:55:08.236947 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.237642 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.238997 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.239411 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.239513 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.240063 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:55:08.240582 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:55:08.240740 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:08.240889 (+ 149us) server_negotiation.cc:299] Negotiation successful
0928 14:55:08.240935 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:55:08.249194 1258 raft_consensus.cc:1261] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364 [term 1 FOLLOWER]: Refusing update from remote peer dc50800f27034f8285e0f86a6633975d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:08.249194 1528 raft_consensus.cc:1261] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a [term 1 FOLLOWER]: Refusing update from remote peer dc50800f27034f8285e0f86a6633975d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:08.250195 1602 consensus_queue.cc:1035] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f50385b4520d4120a7750ad080a72f3a" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:08.250581 1599 consensus_queue.cc:1035] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc4dfbf157014a1b81bfc51989734364" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:08.262463 1605 mvcc.cc:205] Tried to move back new op lower bound from 7334327534578335744 to 7334327534439628800. Current Snapshot: MvccSnapshot[applied={T|T < 7334327534578335744}]
I20260928 14:55:08.277299 1111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:08.277654 1111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:08.281633 1111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcb88158dd1143ba965b17c59179e3ec: Sending DeleteTablet for 3 replicas of tablet 26bb48b5134f43a6ad41d53b051e647c
I20260928 14:55:08.282883 1238 tablet_service.cc:1514] Processing DeleteTablet for tablet 26bb48b5134f43a6ad41d53b051e647c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:08 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:51416
I20260928 14:55:08.282862 1508 tablet_service.cc:1514] Processing DeleteTablet for tablet 26bb48b5134f43a6ad41d53b051e647c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:08 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:44226
I20260928 14:55:08.283595 1374 tablet_service.cc:1514] Processing DeleteTablet for tablet 26bb48b5134f43a6ad41d53b051e647c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:08 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:43454
I20260928 14:55:08.284144 1615 tablet_replica.cc:330] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: stopping tablet replica
I20260928 14:55:08.285117 1615 raft_consensus.cc:2229] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:08.285871 1615 raft_consensus.cc:2258] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:08.287122 1616 tablet_replica.cc:330] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: stopping tablet replica
I20260928 14:55:08.287313 1617 tablet_replica.cc:330] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: stopping tablet replica
I20260928 14:55:08.287716 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 1173
I20260928 14:55:08.287720 1617 raft_consensus.cc:2229] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:55:08.287683 1616 raft_consensus.cc:2229] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:08.288334 1616 raft_consensus.cc:2258] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:08.288446 1617 raft_consensus.cc:2258] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:08.291062 1617 ts_tablet_manager.cc:1905] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:08.291090 1616 ts_tablet_manager.cc:1905] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:08.295373 1615 ts_tablet_manager.cc:1905] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:08.298321 1616 ts_tablet_manager.cc:1918] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:08.298559 1616 log.cc:1196] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/26bb48b5134f43a6ad41d53b051e647c
I20260928 14:55:08.299063 1617 ts_tablet_manager.cc:1918] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:08.299273 1616 ts_tablet_manager.cc:1939] T 26bb48b5134f43a6ad41d53b051e647c P f50385b4520d4120a7750ad080a72f3a: Deleting consensus metadata
I20260928 14:55:08.299316 1617 log.cc:1196] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/26bb48b5134f43a6ad41d53b051e647c
I20260928 14:55:08.299965 1617 ts_tablet_manager.cc:1939] T 26bb48b5134f43a6ad41d53b051e647c P dc50800f27034f8285e0f86a6633975d: Deleting consensus metadata
I20260928 14:55:08.301537 1099 catalog_manager.cc:4841] TS f50385b4520d4120a7750ad080a72f3a (127.14.241.195:42909): tablet 26bb48b5134f43a6ad41d53b051e647c (table test-table [id=967394c9f4a2465eb3927c5e4035a791]) successfully deleted
I20260928 14:55:08.302070 1098 catalog_manager.cc:4841] TS dc50800f27034f8285e0f86a6633975d (127.14.241.194:34963): tablet 26bb48b5134f43a6ad41d53b051e647c (table test-table [id=967394c9f4a2465eb3927c5e4035a791]) successfully deleted
I20260928 14:55:08.306684 1615 ts_tablet_manager.cc:1918] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:08.306862 1615 log.cc:1196] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/26bb48b5134f43a6ad41d53b051e647c
I20260928 14:55:08.307403 1615 ts_tablet_manager.cc:1939] T 26bb48b5134f43a6ad41d53b051e647c P fc4dfbf157014a1b81bfc51989734364: Deleting consensus metadata
Sep 28 14:55:08 dist-test-slave-vk40 krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:55:08.309118 1099 catalog_manager.cc:4841] TS fc4dfbf157014a1b81bfc51989734364 (127.14.241.193:41519): tablet 26bb48b5134f43a6ad41d53b051e647c (table test-table [id=967394c9f4a2465eb3927c5e4035a791]) successfully deleted
I20260928 14:55:08.314889 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.290870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.14.241.193:41519)
0928 14:55:08.291040 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:08.291046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:08.298606 (+ 7560us) server_negotiation.cc:407] Connection header received
0928 14:55:08.298874 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:08.298879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:08.298961 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:08.299057 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:08.300893 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.301822 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.303218 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.303888 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.304052 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.309395 (+ 5343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:08.309406 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:08.309421 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:08.309459 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:08.312235 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:08.312928 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:08.312935 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:08.312937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:08.313071 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:08.313423 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:08.313426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:08.313428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:08.313772 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:08.314046 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:08.314349 (+ 303us) server_negotiation.cc:299] Negotiation successful
0928 14:55:08.314558 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:55:08.315522 1299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:08.404296 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 1173
I20260928 14:55:08.419750 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 1310
Sep 28 14:55:08 dist-test-slave-vk40 krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:55:08.435118 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.420535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43474 (local address 127.14.241.194:34963)
0928 14:55:08.420709 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:08.420714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:08.420784 (+ 70us) server_negotiation.cc:407] Connection header received
0928 14:55:08.420983 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:08.420990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:08.421068 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:08.421154 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:08.422343 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.423176 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.424476 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.424938 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.425044 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.429354 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:08.429368 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:08.429384 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:08.429431 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:08.432327 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:08.433019 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:08.433028 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:08.433030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:08.433186 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:08.433562 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:08.433567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:08.433571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:08.433946 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:08.434260 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:08.434529 (+ 269us) server_negotiation.cc:299] Negotiation successful
0928 14:55:08.434755 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:55:08.435798 1434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:08.547016 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 1310
I20260928 14:55:08.562599 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 1444
Sep 28 14:55:08 dist-test-slave-vk40 krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:55:08.577590 1586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:08.563298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44230 (local address 127.14.241.195:42909)
0928 14:55:08.563506 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:08.563512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:08.563532 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:55:08.563706 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:08.563710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:08.563784 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:08.563871 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:08.565258 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.565987 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.567239 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:08.567629 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:08.567746 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:08.572005 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:08.572020 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:08.572036 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:08.572073 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:08.574895 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:08.575664 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:08.575672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:08.575674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:08.575825 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:08.576200 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:08.576203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:08.576205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:08.576635 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:08.576874 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:08.577136 (+ 262us) server_negotiation.cc:299] Negotiation successful
0928 14:55:08.577318 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 14:55:08.578274 1568 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:08.665346 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 1444
I20260928 14:55:08.679266 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 1080
I20260928 14:55:08.680341 1142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:08.789554 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 1080
2026-09-28T14:55:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4144 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-28T14:55:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:08Z Disabled control of system clock
I20260928 14:55:08.859102 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:35125
--webserver_interface=127.14.241.254
--webserver_port=0
--builtin_ntp_servers=127.14.241.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:35125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:55:09.181407 1626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:09.181694 1626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:09.181785 1626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:09.189074 1626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:09.189183 1626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:09.189218 1626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:09.189275 1626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:09.189316 1626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:09.198550 1626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:35125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:35125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:09.199759 1626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:09.201593 1626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:09.213305 1632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:09.213251 1631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:09.213505 1634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:09.214258 1626 server_base.cc:1034] running on GCE node
I20260928 14:55:09.215090 1626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:09.216837 1626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:09.218039 1626 hybrid_clock.cc:648] HybridClock initialized: now 1790607309217994 us; error 39 us; skew 500 ppm
I20260928 14:55:09.218434 1626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:09.220868 1626 webserver.cc:460] Webserver started at http://127.14.241.254:40405/ using document root <none> and password file <none>
I20260928 14:55:09.221577 1626 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:09.221666 1626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:09.221978 1626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:09.224978 1626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "e20f152023f44a4da3e683e1c028d03c"
format_stamp: "Formatted at 2026-09-28 14:55:09 on dist-test-slave-vk40"
server_key: "113ef094200df26e8e8a32a848dda41b"
server_key_iv: "573e2eaae4208afa2550cc9eadc46a15"
server_key_version: "encryptionkey@0"
I20260928 14:55:09.225775 1626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "e20f152023f44a4da3e683e1c028d03c"
format_stamp: "Formatted at 2026-09-28 14:55:09 on dist-test-slave-vk40"
server_key: "113ef094200df26e8e8a32a848dda41b"
server_key_iv: "573e2eaae4208afa2550cc9eadc46a15"
server_key_version: "encryptionkey@0"
I20260928 14:55:09.231241 1626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:55:09.234905 1640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:09.236353 1626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:55:09.236543 1626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "e20f152023f44a4da3e683e1c028d03c"
format_stamp: "Formatted at 2026-09-28 14:55:09 on dist-test-slave-vk40"
server_key: "113ef094200df26e8e8a32a848dda41b"
server_key_iv: "573e2eaae4208afa2550cc9eadc46a15"
server_key_version: "encryptionkey@0"
I20260928 14:55:09.236768 1626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:09.256915 1626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:09.257831 1626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:09.258109 1626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:09.276062 1626 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:35125
I20260928 14:55:09.276070 1691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:35125 every 8 connection(s)
I20260928 14:55:09.277535 1626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:09.281502 1692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:09.281857 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 1626
I20260928 14:55:09.282024 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:09.282451 15303 external_mini_cluster.cc:1428] Setting key 3b14dabe0a27d844a4a0188262f78e31
I20260928 14:55:09.296161 1692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c: Bootstrap starting.
I20260928 14:55:09.298346 1695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:09.284443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59840 (local address 127.14.241.254:35125)
0928 14:55:09.285591 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:09.285605 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:09.285657 (+ 52us) server_negotiation.cc:407] Connection header received
0928 14:55:09.288140 (+ 2483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:09.288173 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:09.288877 (+ 704us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:09.289371 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:09.289917 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:09.291799 (+ 1882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:09.293227 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:09.294293 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:09.294464 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:09.295522 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:09.295533 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:09.295554 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:09.296262 (+ 708us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:09.296381 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:09.296406 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:09.296524 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:09.296715 (+ 191us) server_negotiation.cc:299] Negotiation successful
0928 14:55:09.297069 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":370,"threads_started":1}
I20260928 14:55:09.299893 1692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:09.300951 1692 log.cc:824] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:09.303575 1692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c: No bootstrap required, opened a new log
I20260928 14:55:09.310947 1692 raft_consensus.cc:348] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20f152023f44a4da3e683e1c028d03c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 35125 } }
I20260928 14:55:09.311303 1692 raft_consensus.cc:374] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:09.311378 1692 raft_consensus.cc:729] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20f152023f44a4da3e683e1c028d03c, State: Initialized, Role: FOLLOWER
I20260928 14:55:09.311910 1692 consensus_queue.cc:260] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20f152023f44a4da3e683e1c028d03c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 35125 } }
I20260928 14:55:09.312109 1692 raft_consensus.cc:388] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:09.312204 1692 raft_consensus.cc:482] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:09.312363 1692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:09.314618 1692 raft_consensus.cc:504] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20f152023f44a4da3e683e1c028d03c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 35125 } }
I20260928 14:55:09.315138 1692 leader_election.cc:302] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e20f152023f44a4da3e683e1c028d03c; no voters:
I20260928 14:55:09.315680 1692 leader_election.cc:288] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:09.315815 1697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:09.316522 1697 raft_consensus.cc:686] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [term 1 LEADER]: Becoming Leader. State: Replica: e20f152023f44a4da3e683e1c028d03c, State: Running, Role: LEADER
I20260928 14:55:09.317006 1697 consensus_queue.cc:237] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20f152023f44a4da3e683e1c028d03c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 35125 } }
I20260928 14:55:09.317692 1692 sys_catalog.cc:564] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:09.319828 1699 sys_catalog.cc:455] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e20f152023f44a4da3e683e1c028d03c. Latest consensus state: current_term: 1 leader_uuid: "e20f152023f44a4da3e683e1c028d03c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20f152023f44a4da3e683e1c028d03c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 35125 } } }
I20260928 14:55:09.319857 1698 sys_catalog.cc:455] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e20f152023f44a4da3e683e1c028d03c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e20f152023f44a4da3e683e1c028d03c" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 35125 } } }
I20260928 14:55:09.320406 1699 sys_catalog.cc:458] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:09.320457 1698 sys_catalog.cc:458] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:09.321535 1703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:09.328078 1703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:09.336973 1703 catalog_manager.cc:1347] Generated new cluster ID: aa8b5d67edc34014a1bdc97bb794cad1
I20260928 14:55:09.337095 1703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:09.370796 1703 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:09.372730 1703 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:09.381881 1703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c: Generated new TSK 0
I20260928 14:55:09.382979 1703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:09.393996 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:35125
--builtin_ntp_servers=127.14.241.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:55:09.719334 1716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:09.719622 1716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:09.719749 1716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:09.727277 1716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:09.727381 1716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:09.727523 1716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:55:09.736811 1716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:35125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:09.738078 1716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:09.739835 1716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:09.752672 1721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:09.752611 1722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:09.752651 1724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:09.753425 1716 server_base.cc:1034] running on GCE node
I20260928 14:55:09.753985 1716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:09.755045 1716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:09.756246 1716 hybrid_clock.cc:648] HybridClock initialized: now 1790607309756201 us; error 45 us; skew 500 ppm
I20260928 14:55:09.756595 1716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:09.758786 1716 webserver.cc:460] Webserver started at http://127.14.241.193:46503/ using document root <none> and password file <none>
I20260928 14:55:09.759451 1716 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:09.759549 1716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:09.759841 1716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:09.762681 1716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "5ece1db639ff4a128a822f10a460d03c"
format_stamp: "Formatted at 2026-09-28 14:55:09 on dist-test-slave-vk40"
server_key: "09c35aa0bb03e8c73ba4fdcd07bbb5a4"
server_key_iv: "4d054250c42eb7f56ef6456f0a2a13a4"
server_key_version: "encryptionkey@0"
I20260928 14:55:09.763448 1716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "5ece1db639ff4a128a822f10a460d03c"
format_stamp: "Formatted at 2026-09-28 14:55:09 on dist-test-slave-vk40"
server_key: "09c35aa0bb03e8c73ba4fdcd07bbb5a4"
server_key_iv: "4d054250c42eb7f56ef6456f0a2a13a4"
server_key_version: "encryptionkey@0"
I20260928 14:55:09.768386 1716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260928 14:55:09.771749 1730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:09.773109 1716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:55:09.773272 1716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "5ece1db639ff4a128a822f10a460d03c"
format_stamp: "Formatted at 2026-09-28 14:55:09 on dist-test-slave-vk40"
server_key: "09c35aa0bb03e8c73ba4fdcd07bbb5a4"
server_key_iv: "4d054250c42eb7f56ef6456f0a2a13a4"
server_key_version: "encryptionkey@0"
I20260928 14:55:09.773463 1716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:09.801218 1716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:09.802090 1716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:09.802376 1716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:09.803324 1716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:09.804811 1716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:09.804908 1716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:09.805020 1716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:09.805085 1716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:09.834668 1716 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:39637
I20260928 14:55:09.834734 1842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:39637 every 8 connection(s)
I20260928 14:55:09.835978 1716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:55:09.840608 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 1716
I20260928 14:55:09.840767 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:55:09.841158 15303 external_mini_cluster.cc:1428] Setting key 23e9708a9129c2ed118ed7e72d919f8e
I20260928 14:55:09.844125 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:35125
--builtin_ntp_servers=127.14.241.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:55:09.848853 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:09.838382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:40223 (local address 127.14.241.254:35125)
0928 14:55:09.838805 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:09.838812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:09.840070 (+ 1258us) server_negotiation.cc:407] Connection header received
0928 14:55:09.841999 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:09.842005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:09.842089 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:09.842184 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:09.843461 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:09.844518 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:09.846045 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:09.846561 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:09.846691 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:09.848021 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:09.848028 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:09.848033 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:09.848056 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:09.848102 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:09.848116 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:09.848197 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:09.848417 (+ 220us) server_negotiation.cc:299] Negotiation successful
0928 14:55:09.848520 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":189,"threads_started":1}
I20260928 14:55:09.849771 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:09.839010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:35125 (local address 127.14.241.193:40223)
0928 14:55:09.839913 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:09.839957 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:09.841748 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:09.842481 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:09.842494 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:09.842632 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:09.843285 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:09.843304 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:09.844659 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:09.844663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:09.845905 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:09.845917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:09.846752 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:09.846757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:09.847015 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:09.847683 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:55:09.847687 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:09.847738 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:55:09.847828 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:09.848205 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:09.848210 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:09.848215 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:55:09.848399 (+ 184us) client_negotiation.cc:240] Negotiation successful
0928 14:55:09.848714 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":274,"threads_started":1}
I20260928 14:55:09.851285 1843 heartbeater.cc:347] Connected to a master server at 127.14.241.254:35125
I20260928 14:55:09.851609 1843 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:09.852453 1843 heartbeater.cc:511] Master 127.14.241.254:35125 requested a full tablet report, sending...
I20260928 14:55:09.854620 1657 ts_manager.cc:194] Registered new tserver with Master: 5ece1db639ff4a128a822f10a460d03c (127.14.241.193:39637)
I20260928 14:55:09.856627 1657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.193:40223
W20260928 14:55:10.169003 1847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:10.169288 1847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:10.169373 1847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:10.176635 1847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:10.176772 1847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:10.176903 1847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:55:10.186183 1847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:35125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:10.187443 1847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:10.189198 1847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:10.202000 1852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:10.202066 1853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:10.202036 1855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:10.202631 1847 server_base.cc:1034] running on GCE node
I20260928 14:55:10.203285 1847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:10.204298 1847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:10.205513 1847 hybrid_clock.cc:648] HybridClock initialized: now 1790607310205456 us; error 51 us; skew 500 ppm
I20260928 14:55:10.205835 1847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:10.208166 1847 webserver.cc:460] Webserver started at http://127.14.241.194:39721/ using document root <none> and password file <none>
I20260928 14:55:10.208920 1847 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:10.209012 1847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:10.209316 1847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:10.212191 1847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "a564ce3bda5344e286f7ae497584feae"
format_stamp: "Formatted at 2026-09-28 14:55:10 on dist-test-slave-vk40"
server_key: "9a291d85e6ddfd8535cc34ce2515badd"
server_key_iv: "d1a7e8e4877edb6c71a65d8af629bc1c"
server_key_version: "encryptionkey@0"
I20260928 14:55:10.213011 1847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "a564ce3bda5344e286f7ae497584feae"
format_stamp: "Formatted at 2026-09-28 14:55:10 on dist-test-slave-vk40"
server_key: "9a291d85e6ddfd8535cc34ce2515badd"
server_key_iv: "d1a7e8e4877edb6c71a65d8af629bc1c"
server_key_version: "encryptionkey@0"
I20260928 14:55:10.218161 1847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:10.221751 1861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:10.223367 1847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:55:10.223507 1847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "a564ce3bda5344e286f7ae497584feae"
format_stamp: "Formatted at 2026-09-28 14:55:10 on dist-test-slave-vk40"
server_key: "9a291d85e6ddfd8535cc34ce2515badd"
server_key_iv: "d1a7e8e4877edb6c71a65d8af629bc1c"
server_key_version: "encryptionkey@0"
I20260928 14:55:10.223692 1847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:10.244254 1847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:10.245100 1847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:10.245373 1847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:10.246315 1847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:10.247872 1847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:10.247958 1847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:10.248039 1847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:10.248090 1847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:10.276202 1847 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:46173
I20260928 14:55:10.276335 1973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:46173 every 8 connection(s)
I20260928 14:55:10.277575 1847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:55:10.280041 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 1847
I20260928 14:55:10.280197 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:55:10.280709 15303 external_mini_cluster.cc:1428] Setting key b00337afccf7d7af1fe61ee40f3f90f7
I20260928 14:55:10.283517 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:35125
--builtin_ntp_servers=127.14.241.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:55:10.293054 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.280314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:33653 (local address 127.14.241.254:35125)
0928 14:55:10.280539 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:10.280546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:10.282143 (+ 1597us) server_negotiation.cc:407] Connection header received
0928 14:55:10.284726 (+ 2583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:10.284733 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:10.284810 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:10.284921 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:10.286318 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.287315 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.289732 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.290374 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.290468 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.292221 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:10.292226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:10.292230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:10.292249 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:10.292307 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:10.292322 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:10.292418 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:10.292621 (+ 203us) server_negotiation.cc:299] Negotiation successful
0928 14:55:10.292723 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 14:55:10.294086 1976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.280897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:35125 (local address 127.14.241.194:33653)
0928 14:55:10.281985 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:10.282031 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:10.284421 (+ 2390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:10.285161 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:10.285172 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:10.285294 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:10.286121 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.286152 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.287463 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.287469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.289561 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.289579 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.290766 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.290773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.291082 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.291874 (+ 792us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:55:10.291878 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:10.291930 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:55:10.292033 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:10.292420 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:10.292432 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:10.292441 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 14:55:10.292597 (+ 156us) client_negotiation.cc:240] Negotiation successful
0928 14:55:10.292988 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":325,"threads_started":1}
I20260928 14:55:10.295603 1974 heartbeater.cc:347] Connected to a master server at 127.14.241.254:35125
I20260928 14:55:10.295944 1974 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:10.296742 1974 heartbeater.cc:511] Master 127.14.241.254:35125 requested a full tablet report, sending...
I20260928 14:55:10.298158 1657 ts_manager.cc:194] Registered new tserver with Master: a564ce3bda5344e286f7ae497584feae (127.14.241.194:46173)
I20260928 14:55:10.299124 1657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.194:33653
W20260928 14:55:10.605082 1977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:10.605367 1977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:10.605451 1977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:10.612919 1977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:10.613029 1977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:10.613135 1977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:55:10.622339 1977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:35125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:10.623540 1977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:10.625329 1977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:10.637850 1982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:10.637877 1983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:10.637923 1985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:10.638523 1977 server_base.cc:1034] running on GCE node
I20260928 14:55:10.639184 1977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:10.640169 1977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:10.641415 1977 hybrid_clock.cc:648] HybridClock initialized: now 1790607310641334 us; error 58 us; skew 500 ppm
I20260928 14:55:10.641773 1977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:10.643936 1977 webserver.cc:460] Webserver started at http://127.14.241.195:39311/ using document root <none> and password file <none>
I20260928 14:55:10.644656 1977 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:10.644752 1977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:10.645047 1977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:10.647943 1977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "5359a0db46884b1088b1056bf176ca10"
format_stamp: "Formatted at 2026-09-28 14:55:10 on dist-test-slave-vk40"
server_key: "767711f66b01a2c56fbfb9e0f50c1c56"
server_key_iv: "d56f7e45eaddbeee073b3895d3bbc565"
server_key_version: "encryptionkey@0"
I20260928 14:55:10.648773 1977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "5359a0db46884b1088b1056bf176ca10"
format_stamp: "Formatted at 2026-09-28 14:55:10 on dist-test-slave-vk40"
server_key: "767711f66b01a2c56fbfb9e0f50c1c56"
server_key_iv: "d56f7e45eaddbeee073b3895d3bbc565"
server_key_version: "encryptionkey@0"
I20260928 14:55:10.653852 1977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260928 14:55:10.657202 1991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:10.658511 1977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:55:10.658654 1977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "5359a0db46884b1088b1056bf176ca10"
format_stamp: "Formatted at 2026-09-28 14:55:10 on dist-test-slave-vk40"
server_key: "767711f66b01a2c56fbfb9e0f50c1c56"
server_key_iv: "d56f7e45eaddbeee073b3895d3bbc565"
server_key_version: "encryptionkey@0"
I20260928 14:55:10.658854 1977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:10.698374 1977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:10.699229 1977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:10.699512 1977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:10.700495 1977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:10.701962 1977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:10.702049 1977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:10.702157 1977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:10.702229 1977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:10.731917 1977 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:35629
I20260928 14:55:10.731972 2103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:35629 every 8 connection(s)
I20260928 14:55:10.733202 1977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:55:10.741539 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 1977
I20260928 14:55:10.741722 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:55:10.742126 15303 external_mini_cluster.cc:1428] Setting key 5c5d3bdc412b88ef459593cadf26367c
I20260928 14:55:10.745640 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.735630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:46605 (local address 127.14.241.254:35125)
0928 14:55:10.735799 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:10.735804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:10.737235 (+ 1431us) server_negotiation.cc:407] Connection header received
0928 14:55:10.739045 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:10.739051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:10.739131 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:10.739217 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:10.740305 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.741113 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.742868 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.743409 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.743517 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.744891 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:10.744896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:10.744899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:10.744918 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:10.744948 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:10.744958 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:10.745030 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:10.745238 (+ 208us) server_negotiation.cc:299] Negotiation successful
0928 14:55:10.745334 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 14:55:10.746595 2106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.736150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:35125 (local address 127.14.241.195:46605)
0928 14:55:10.737071 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:10.737107 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:10.738768 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:10.739460 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:10.739469 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:10.739568 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:10.740120 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.740136 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.741263 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.741267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.742729 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.742744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.743619 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.743626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.743848 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.744563 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:55:10.744566 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:10.744620 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:55:10.744710 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:10.745039 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:10.745050 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:10.745059 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 14:55:10.745210 (+ 151us) client_negotiation.cc:240] Negotiation successful
0928 14:55:10.745611 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":290,"threads_started":1}
I20260928 14:55:10.747809 2104 heartbeater.cc:347] Connected to a master server at 127.14.241.254:35125
I20260928 14:55:10.748081 2104 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:10.748890 2104 heartbeater.cc:511] Master 127.14.241.254:35125 requested a full tablet report, sending...
I20260928 14:55:10.750159 1657 ts_manager.cc:194] Registered new tserver with Master: 5359a0db46884b1088b1056bf176ca10 (127.14.241.195:35629)
I20260928 14:55:10.750913 1657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.195:46605
I20260928 14:55:10.756944 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:55:10.766355 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.760866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59856 (local address 127.14.241.254:35125)
0928 14:55:10.761104 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:10.761110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:10.761410 (+ 300us) server_negotiation.cc:407] Connection header received
0928 14:55:10.761541 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:10.761545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:10.761615 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:10.761706 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:10.762130 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.762872 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.764214 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.764656 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.764744 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.765697 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:10.765702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:10.765705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:10.765724 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:10.765755 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:10.765766 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:10.765885 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:10.766006 (+ 121us) server_negotiation.cc:299] Negotiation successful
0928 14:55:10.766057 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 14:55:10.770056 1657 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:55:10.776767 1657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:55:10.809706 2120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.788648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:35629 (local address 127.0.0.1:34044)
0928 14:55:10.798590 (+ 9942us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:10.798654 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:10.798865 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:10.803043 (+ 4178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:10.803048 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:10.803070 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:10.803461 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.803476 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.805293 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.805299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.806922 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.806935 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.808800 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.808807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.808987 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.809040 (+ 53us) client_negotiation.cc:769] Sending connection context
0928 14:55:10.809163 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 14:55:10.809370 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9617,"mutex_wait_us":1,"thread_start_us":12143,"threads_started":1}
I20260928 14:55:10.810061 2121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.791201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39480 (local address 127.14.241.193:39637)
0928 14:55:10.799485 (+ 8284us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:10.799495 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:10.802546 (+ 3051us) server_negotiation.cc:407] Connection header received
0928 14:55:10.802665 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:10.802673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:10.802906 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:10.803094 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:10.803872 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.805269 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.807688 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.808980 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.809135 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.809332 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:10.809555 (+ 223us) server_negotiation.cc:299] Negotiation successful
0928 14:55:10.809727 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8091,"thread_start_us":7953,"threads_started":1}
I20260928 14:55:10.810477 2116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.788647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:46173 (local address 127.0.0.1:45634)
0928 14:55:10.801123 (+ 12476us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:10.801142 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:10.801282 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:10.802120 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:10.802129 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:10.802174 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:10.802627 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.802641 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.804662 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.804667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.807219 (+ 2552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.807237 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.809787 (+ 2550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.809795 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.809977 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.810006 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 14:55:10.810066 (+ 60us) client_negotiation.cc:240] Negotiation successful
0928 14:55:10.810188 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12317,"mutex_wait_us":285,"thread_start_us":14426,"threads_started":1}
I20260928 14:55:10.810773 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.788875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34044 (local address 127.14.241.195:35629)
0928 14:55:10.802577 (+ 13702us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:10.802586 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:10.802612 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:55:10.802686 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:10.802691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:10.802870 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:10.803026 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:10.803654 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.805136 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.807112 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.808492 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.808690 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.808921 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:10.810286 (+ 1365us) server_negotiation.cc:299] Negotiation successful
0928 14:55:10.810472 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13533,"thread_start_us":13394,"threads_started":1}
I20260928 14:55:10.811483 2117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.788742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45634 (local address 127.14.241.194:46173)
0928 14:55:10.800735 (+ 11993us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:10.800744 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:10.801427 (+ 683us) server_negotiation.cc:407] Connection header received
0928 14:55:10.801550 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:10.801560 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:10.801862 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:10.802838 (+ 976us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:10.803033 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.804516 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.807408 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.809585 (+ 2177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.810752 (+ 1167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.810972 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:10.811037 (+ 65us) server_negotiation.cc:299] Negotiation successful
0928 14:55:10.811198 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11825,"thread_start_us":11718,"threads_started":1}
I20260928 14:55:10.812045 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.790644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39637 (local address 127.0.0.1:39480)
0928 14:55:10.802261 (+ 11617us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:10.802280 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:10.802414 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:10.803299 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:10.803304 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:10.803329 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:10.803712 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.803720 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.805396 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.805404 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.807533 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.807543 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.809180 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.809187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.809371 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.809405 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 14:55:10.809502 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 14:55:10.809619 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11494,"mutex_wait_us":32,"thread_start_us":7449,"threads_started":1}
I20260928 14:55:10.814674 1909 tablet_service.cc:1467] Processing CreateTablet for tablet 5e7715b944df45f9beb6d4f984d2b16f (DEFAULT_TABLE table=test-table [id=8525d3ff4a3945ffb14fd2bdba7a67d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:10.814749 2039 tablet_service.cc:1467] Processing CreateTablet for tablet 5e7715b944df45f9beb6d4f984d2b16f (DEFAULT_TABLE table=test-table [id=8525d3ff4a3945ffb14fd2bdba7a67d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:10.815850 1909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e7715b944df45f9beb6d4f984d2b16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:10.815645 1778 tablet_service.cc:1467] Processing CreateTablet for tablet 5e7715b944df45f9beb6d4f984d2b16f (DEFAULT_TABLE table=test-table [id=8525d3ff4a3945ffb14fd2bdba7a67d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:10.816133 2039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e7715b944df45f9beb6d4f984d2b16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:10.816900 1778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e7715b944df45f9beb6d4f984d2b16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:10.827301 2122 tablet_bootstrap.cc:492] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae: Bootstrap starting.
I20260928 14:55:10.830978 2123 tablet_bootstrap.cc:492] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10: Bootstrap starting.
I20260928 14:55:10.832091 2122 tablet_bootstrap.cc:654] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:10.833346 2122 log.cc:824] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:10.834841 2123 tablet_bootstrap.cc:654] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:10.836094 2123 log.cc:824] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:10.836977 2124 tablet_bootstrap.cc:492] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c: Bootstrap starting.
I20260928 14:55:10.839950 2124 tablet_bootstrap.cc:654] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:10.841125 2124 log.cc:824] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:10.847170 2123 tablet_bootstrap.cc:492] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10: No bootstrap required, opened a new log
I20260928 14:55:10.847487 2123 ts_tablet_manager.cc:1397] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.000s
I20260928 14:55:10.847935 2122 tablet_bootstrap.cc:492] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae: No bootstrap required, opened a new log
I20260928 14:55:10.848201 2122 ts_tablet_manager.cc:1397] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.007s
I20260928 14:55:10.848399 2124 tablet_bootstrap.cc:492] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c: No bootstrap required, opened a new log
I20260928 14:55:10.848658 2124 ts_tablet_manager.cc:1397] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260928 14:55:10.855849 2122 raft_consensus.cc:348] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a564ce3bda5344e286f7ae497584feae" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46173 } } peers { permanent_uuid: "5359a0db46884b1088b1056bf176ca10" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35629 } } peers { permanent_uuid: "5ece1db639ff4a128a822f10a460d03c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39637 } }
I20260928 14:55:10.856241 2122 raft_consensus.cc:374] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:10.856356 2122 raft_consensus.cc:729] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a564ce3bda5344e286f7ae497584feae, State: Initialized, Role: FOLLOWER
I20260928 14:55:10.856956 2122 consensus_queue.cc:260] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a564ce3bda5344e286f7ae497584feae" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46173 } } peers { permanent_uuid: "5359a0db46884b1088b1056bf176ca10" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35629 } } peers { permanent_uuid: "5ece1db639ff4a128a822f10a460d03c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39637 } }
I20260928 14:55:10.858006 1974 heartbeater.cc:503] Master 127.14.241.254:35125 was elected leader, sending a full tablet report...
I20260928 14:55:10.857883 2123 raft_consensus.cc:348] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a564ce3bda5344e286f7ae497584feae" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46173 } } peers { permanent_uuid: "5359a0db46884b1088b1056bf176ca10" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35629 } } peers { permanent_uuid: "5ece1db639ff4a128a822f10a460d03c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39637 } }
I20260928 14:55:10.858286 2123 raft_consensus.cc:374] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:10.858372 2123 raft_consensus.cc:729] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5359a0db46884b1088b1056bf176ca10, State: Initialized, Role: FOLLOWER
I20260928 14:55:10.858726 2122 ts_tablet_manager.cc:1428] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260928 14:55:10.858703 2124 raft_consensus.cc:348] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a564ce3bda5344e286f7ae497584feae" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46173 } } peers { permanent_uuid: "5359a0db46884b1088b1056bf176ca10" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35629 } } peers { permanent_uuid: "5ece1db639ff4a128a822f10a460d03c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39637 } }
I20260928 14:55:10.859092 2124 raft_consensus.cc:374] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:10.859194 2124 raft_consensus.cc:729] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ece1db639ff4a128a822f10a460d03c, State: Initialized, Role: FOLLOWER
I20260928 14:55:10.859100 2123 consensus_queue.cc:260] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a564ce3bda5344e286f7ae497584feae" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46173 } } peers { permanent_uuid: "5359a0db46884b1088b1056bf176ca10" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35629 } } peers { permanent_uuid: "5ece1db639ff4a128a822f10a460d03c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39637 } }
I20260928 14:55:10.859606 1843 heartbeater.cc:503] Master 127.14.241.254:35125 was elected leader, sending a full tablet report...
I20260928 14:55:10.859865 2124 consensus_queue.cc:260] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a564ce3bda5344e286f7ae497584feae" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46173 } } peers { permanent_uuid: "5359a0db46884b1088b1056bf176ca10" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35629 } } peers { permanent_uuid: "5ece1db639ff4a128a822f10a460d03c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39637 } }
I20260928 14:55:10.860889 2104 heartbeater.cc:503] Master 127.14.241.254:35125 was elected leader, sending a full tablet report...
I20260928 14:55:10.862854 2123 ts_tablet_manager.cc:1428] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260928 14:55:10.863984 2124 ts_tablet_manager.cc:1428] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260928 14:55:10.899922 2130 raft_consensus.cc:482] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:55:10.900223 2130 raft_consensus.cc:504] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a564ce3bda5344e286f7ae497584feae" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46173 } } peers { permanent_uuid: "5359a0db46884b1088b1056bf176ca10" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35629 } } peers { permanent_uuid: "5ece1db639ff4a128a822f10a460d03c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39637 } }
I20260928 14:55:10.902578 2130 leader_election.cc:288] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a564ce3bda5344e286f7ae497584feae (127.14.241.194:46173), 5ece1db639ff4a128a822f10a460d03c (127.14.241.193:39637)
I20260928 14:55:10.907447 2121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.902456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:59201 (local address 127.14.241.193:39637)
0928 14:55:10.902653 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:10.902658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:10.902858 (+ 200us) server_negotiation.cc:407] Connection header received
0928 14:55:10.902938 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:10.902943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:10.903029 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:10.903107 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:10.903542 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.904209 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.905853 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.906598 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.906755 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.906823 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:10.907061 (+ 238us) server_negotiation.cc:299] Negotiation successful
0928 14:55:10.907183 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:55:10.907481 2106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.902336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39637 (local address 127.14.241.195:59201)
0928 14:55:10.902654 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:10.902672 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:10.902802 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:10.903171 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:10.903175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:10.903202 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:10.903405 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.903414 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.904362 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.904367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.905703 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.905715 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.906748 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.906753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.906924 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.906947 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 14:55:10.907024 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:55:10.907153 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":19,"spinlock_wait_cycles":44800}
I20260928 14:55:10.908129 2117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.902459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:44777 (local address 127.14.241.194:46173)
0928 14:55:10.902802 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:10.902807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:10.903259 (+ 452us) server_negotiation.cc:407] Connection header received
0928 14:55:10.903432 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:10.903436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:10.903505 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:10.903582 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:10.904076 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.904780 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.906504 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.907263 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.907400 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.907465 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:10.907808 (+ 343us) server_negotiation.cc:299] Negotiation successful
0928 14:55:10.907945 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221}
I20260928 14:55:10.908176 2132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.902446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:46173 (local address 127.14.241.195:44777)
0928 14:55:10.903154 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:10.903169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:10.903302 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:10.903609 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:10.903613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:10.903634 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:10.903935 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.903943 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.904923 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.904928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.906320 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:10.906332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.907425 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.907431 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:10.907618 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.907650 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 14:55:10.907763 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 14:55:10.907880 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":35,"thread_start_us":443,"threads_started":1}
I20260928 14:55:10.908566 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7715b944df45f9beb6d4f984d2b16f" candidate_uuid: "5359a0db46884b1088b1056bf176ca10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ece1db639ff4a128a822f10a460d03c" is_pre_election: true
I20260928 14:55:10.909072 1798 raft_consensus.cc:2454] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5359a0db46884b1088b1056bf176ca10 in term 0.
I20260928 14:55:10.909111 1929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7715b944df45f9beb6d4f984d2b16f" candidate_uuid: "5359a0db46884b1088b1056bf176ca10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a564ce3bda5344e286f7ae497584feae" is_pre_election: true
I20260928 14:55:10.909586 1929 raft_consensus.cc:2454] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5359a0db46884b1088b1056bf176ca10 in term 0.
I20260928 14:55:10.909641 1994 leader_election.cc:302] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5359a0db46884b1088b1056bf176ca10, 5ece1db639ff4a128a822f10a460d03c; no voters:
I20260928 14:55:10.910007 2130 raft_consensus.cc:2781] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:55:10.910133 2130 raft_consensus.cc:482] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:55:10.910228 2130 raft_consensus.cc:3037] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:10.912413 2130 raft_consensus.cc:504] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a564ce3bda5344e286f7ae497584feae" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46173 } } peers { permanent_uuid: "5359a0db46884b1088b1056bf176ca10" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35629 } } peers { permanent_uuid: "5ece1db639ff4a128a822f10a460d03c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39637 } }
I20260928 14:55:10.913012 2130 leader_election.cc:288] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [CANDIDATE]: Term 1 election: Requested vote from peers a564ce3bda5344e286f7ae497584feae (127.14.241.194:46173), 5ece1db639ff4a128a822f10a460d03c (127.14.241.193:39637)
I20260928 14:55:10.913385 1929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7715b944df45f9beb6d4f984d2b16f" candidate_uuid: "5359a0db46884b1088b1056bf176ca10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a564ce3bda5344e286f7ae497584feae"
I20260928 14:55:10.913493 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7715b944df45f9beb6d4f984d2b16f" candidate_uuid: "5359a0db46884b1088b1056bf176ca10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ece1db639ff4a128a822f10a460d03c"
I20260928 14:55:10.913570 1929 raft_consensus.cc:3037] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:10.913697 1798 raft_consensus.cc:3037] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:10.915779 1929 raft_consensus.cc:2454] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5359a0db46884b1088b1056bf176ca10 in term 1.
I20260928 14:55:10.915841 1798 raft_consensus.cc:2454] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5359a0db46884b1088b1056bf176ca10 in term 1.
I20260928 14:55:10.916180 1995 leader_election.cc:302] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5359a0db46884b1088b1056bf176ca10, a564ce3bda5344e286f7ae497584feae; no voters:
I20260928 14:55:10.916517 2130 raft_consensus.cc:2781] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:10.916940 2130 raft_consensus.cc:686] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 1 LEADER]: Becoming Leader. State: Replica: 5359a0db46884b1088b1056bf176ca10, State: Running, Role: LEADER
I20260928 14:55:10.917433 2130 consensus_queue.cc:237] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a564ce3bda5344e286f7ae497584feae" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46173 } } peers { permanent_uuid: "5359a0db46884b1088b1056bf176ca10" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35629 } } peers { permanent_uuid: "5ece1db639ff4a128a822f10a460d03c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39637 } }
I20260928 14:55:10.921832 1656 catalog_manager.cc:5495] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5359a0db46884b1088b1056bf176ca10 (127.14.241.195). New cstate: current_term: 1 leader_uuid: "5359a0db46884b1088b1056bf176ca10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a564ce3bda5344e286f7ae497584feae" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5359a0db46884b1088b1056bf176ca10" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ece1db639ff4a128a822f10a460d03c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39637 } health_report { overall_health: UNKNOWN } } }
I20260928 14:55:10.936790 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.931955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34060 (local address 127.14.241.195:35629)
0928 14:55:10.932114 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:10.932119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:10.932318 (+ 199us) server_negotiation.cc:407] Connection header received
0928 14:55:10.932506 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:10.932510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:10.932589 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:10.932703 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:55:10.933001 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.933706 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.934960 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.935347 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.935429 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.935846 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:55:10.936350 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:55:10.936484 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:10.936573 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 14:55:10.936608 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:55:10.944497 1929 raft_consensus.cc:1261] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae [term 1 FOLLOWER]: Refusing update from remote peer 5359a0db46884b1088b1056bf176ca10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:10.944499 1798 raft_consensus.cc:1261] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c [term 1 FOLLOWER]: Refusing update from remote peer 5359a0db46884b1088b1056bf176ca10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:10.945395 2130 consensus_queue.cc:1035] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a564ce3bda5344e286f7ae497584feae" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:10.945670 2133 consensus_queue.cc:1035] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ece1db639ff4a128a822f10a460d03c" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:10.956135 2136 mvcc.cc:205] Tried to move back new op lower bound from 7334327545619406848 to 7334327545521717248. Current Snapshot: MvccSnapshot[applied={T|T < 7334327545619406848}]
I20260928 14:55:10.960837 2138 mvcc.cc:205] Tried to move back new op lower bound from 7334327545619406848 to 7334327545521717248. Current Snapshot: MvccSnapshot[applied={T|T < 7334327545619406848}]
I20260928 14:55:10.972828 1657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:10.973119 1657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:10.976888 1657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e20f152023f44a4da3e683e1c028d03c: Sending DeleteTablet for 3 replicas of tablet 5e7715b944df45f9beb6d4f984d2b16f
I20260928 14:55:10.978034 2039 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e7715b944df45f9beb6d4f984d2b16f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:10 UTC) from {username='slave'} at 127.0.0.1:34044
I20260928 14:55:10.978343 1778 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e7715b944df45f9beb6d4f984d2b16f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:10 UTC) from {username='slave'} at 127.0.0.1:39480
I20260928 14:55:10.978731 1909 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e7715b944df45f9beb6d4f984d2b16f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:10 UTC) from {username='slave'} at 127.0.0.1:45634
I20260928 14:55:10.979543 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 1716
I20260928 14:55:10.988792 2147 tablet_replica.cc:330] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c: stopping tablet replica
I20260928 14:55:10.989326 2147 raft_consensus.cc:2229] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:10.990029 2147 raft_consensus.cc:2258] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:10.991091 2148 tablet_replica.cc:330] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae: stopping tablet replica
I20260928 14:55:10.991578 2148 raft_consensus.cc:2229] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:10.992105 2148 raft_consensus.cc:2258] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:10.992758 2147 ts_tablet_manager.cc:1905] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:10.993093 2146 tablet_replica.cc:330] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10: stopping tablet replica
I20260928 14:55:10.993484 2146 raft_consensus.cc:2229] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:55:10.994251 2146 raft_consensus.cc:2258] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:10.994678 2148 ts_tablet_manager.cc:1905] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:10.999081 2121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:10.983866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39494 (local address 127.14.241.193:39637)
0928 14:55:10.984060 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:10.984067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:10.988468 (+ 4401us) server_negotiation.cc:407] Connection header received
0928 14:55:10.988892 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:10.988898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:10.989027 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:10.989108 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:10.989727 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.990879 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.994590 (+ 3711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:10.995260 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:10.995366 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:10.997004 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:10.997015 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:10.997031 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:10.997783 (+ 752us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:10.997839 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:10.997877 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:10.997969 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:10.998551 (+ 582us) server_negotiation.cc:299] Negotiation successful
0928 14:55:10.998686 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 14:55:11.000206 1838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:11.001523 2146 ts_tablet_manager.cc:1905] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:11.002673 2148 ts_tablet_manager.cc:1918] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:11.002839 2148 log.cc:1196] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/5e7715b944df45f9beb6d4f984d2b16f
I20260928 14:55:11.003355 2148 ts_tablet_manager.cc:1939] T 5e7715b944df45f9beb6d4f984d2b16f P a564ce3bda5344e286f7ae497584feae: Deleting consensus metadata
I20260928 14:55:11.005271 1644 catalog_manager.cc:4841] TS a564ce3bda5344e286f7ae497584feae (127.14.241.194:46173): tablet 5e7715b944df45f9beb6d4f984d2b16f (table test-table [id=8525d3ff4a3945ffb14fd2bdba7a67d3]) successfully deleted
I20260928 14:55:11.009313 2146 ts_tablet_manager.cc:1918] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:11.009473 2146 log.cc:1196] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/5e7715b944df45f9beb6d4f984d2b16f
I20260928 14:55:11.009915 2146 ts_tablet_manager.cc:1939] T 5e7715b944df45f9beb6d4f984d2b16f P 5359a0db46884b1088b1056bf176ca10: Deleting consensus metadata
I20260928 14:55:11.011504 1642 catalog_manager.cc:4841] TS 5359a0db46884b1088b1056bf176ca10 (127.14.241.195:35629): tablet 5e7715b944df45f9beb6d4f984d2b16f (table test-table [id=8525d3ff4a3945ffb14fd2bdba7a67d3]) successfully deleted
I20260928 14:55:11.126993 2147 ts_tablet_manager.cc:1918] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:11.127245 2147 log.cc:1196] T 5e7715b944df45f9beb6d4f984d2b16f P 5ece1db639ff4a128a822f10a460d03c: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/5e7715b944df45f9beb6d4f984d2b16f
I20260928 14:55:11.127662 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 1716
I20260928 14:55:11.142874 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 1847
W20260928 14:55:11.142922 1643 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.241.193:39637 (error 108)
I20260928 14:55:11.145040 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:11.144521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39637 (local address 127.0.0.1:39510)
0928 14:55:11.144721 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:11.144864 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:39637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260928 14:55:11.145349 1643 catalog_manager.cc:4584] TS 5ece1db639ff4a128a822f10a460d03c (127.14.241.193:39637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e7715b944df45f9beb6d4f984d2b16f: Network error: Client connection negotiation failed: client connection to 127.14.241.193:39637: connect: Connection refused (error 111)
I20260928 14:55:11.150384 2117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:11.144187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45640 (local address 127.14.241.194:46173)
0928 14:55:11.144402 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:11.144408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:11.144428 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:55:11.144498 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:11.144503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:11.144628 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:11.144739 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:11.145170 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:11.146089 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:11.147376 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:11.147781 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:11.147906 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:11.149013 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:11.149022 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:11.149036 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:11.149652 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:11.149700 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:11.149722 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:11.149824 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:11.149957 (+ 133us) server_negotiation.cc:299] Negotiation successful
0928 14:55:11.150071 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 14:55:11.151088 1969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:11.199081 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:11.198505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39637 (local address 127.0.0.1:39522)
0928 14:55:11.198758 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:11.198883 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:39637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260928 14:55:11.201282 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:11.199849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39637 (local address 127.0.0.1:39530)
0928 14:55:11.201023 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:11.201118 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:39637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 14:55:11.241056 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 1847
I20260928 14:55:11.255415 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 1977
I20260928 14:55:11.262219 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:11.261769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39637 (local address 127.0.0.1:39536)
0928 14:55:11.261964 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:11.262040 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:39637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 14:55:11.263300 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:11.262845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39637 (local address 127.0.0.1:39548)
0928 14:55:11.263035 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:11.263129 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:39637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260928 14:55:11.264024 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:11.256233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.14.241.195:35629)
0928 14:55:11.256464 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:11.256470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:11.256493 (+ 23us) server_negotiation.cc:407] Connection header received
0928 14:55:11.256578 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:11.256583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:11.256705 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:11.256821 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:11.257185 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:11.258396 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:11.259642 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:11.260463 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:11.260599 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:11.261621 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:11.261633 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:11.261652 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:11.262588 (+ 936us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:11.262656 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:11.262687 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:11.262792 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:11.263555 (+ 763us) server_negotiation.cc:299] Negotiation successful
0928 14:55:11.263664 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260928 14:55:11.264921 2099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:11.343954 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:11.343440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39637 (local address 127.0.0.1:39560)
0928 14:55:11.343651 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:11.343806 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:39637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260928 14:55:11.345031 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:11.344638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39637 (local address 127.0.0.1:39572)
0928 14:55:11.344837 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:11.344913 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:39637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260928 14:55:11.400810 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 1977
I20260928 14:55:11.417680 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 1626
I20260928 14:55:11.418836 1687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:11.528121 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 1626
I20260928 14:55:11.528172 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:11.527668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39637 (local address 127.0.0.1:39586)
0928 14:55:11.527842 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:11.527964 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:39637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
2026-09-28T14:55:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2737 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:11 dist-test-slave-vk40 krb5kdc[2157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:11 dist-test-slave-vk40 krb5kdc[2157](info): set up 2 sockets
Sep 28 14:55:11 dist-test-slave-vk40 krb5kdc[2157](info): commencing operation
krb5kdc: starting...
W20260928 14:55:12.559213 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:12 dist-test-slave-vk40 krb5kdc[2157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:55:12.803541 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:34703
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:34703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:13.125780 2170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:13.126093 2170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:13.126219 2170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:13.133592 2170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:13.133725 2170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:13.133805 2170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:13.133880 2170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:13.133939 2170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:13.144001 2170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:34703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:34703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:13.145391 2170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:13.146703 2170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:13.158516 2175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:13.158449 2176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:13.158469 2178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:13.159260 2170 server_base.cc:1034] running on GCE node
I20260928 14:55:13.160040 2170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:13.161645 2170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:13.162864 2170 hybrid_clock.cc:648] HybridClock initialized: now 1790607313162808 us; error 51 us; skew 500 ppm
I20260928 14:55:13.163239 2170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:13 dist-test-slave-vk40 krb5kdc[2157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607313, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:13.172529 2170 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:55:13.174307 2170 webserver.cc:460] Webserver started at http://127.14.241.254:35805/ using document root <none> and password file <none>
I20260928 14:55:13.175097 2170 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:13.175233 2170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:13.175557 2170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:13.178902 2170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "5a5da690d2ba4bad971e31ec8c3aa325"
format_stamp: "Formatted at 2026-09-28 14:55:13 on dist-test-slave-vk40"
server_key: "eb86637ad34b3523ec91ff89296e8723"
server_key_iv: "3d4a515ade0031a464e6bb3603028e61"
server_key_version: "encryptionkey@0"
I20260928 14:55:13.179752 2170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "5a5da690d2ba4bad971e31ec8c3aa325"
format_stamp: "Formatted at 2026-09-28 14:55:13 on dist-test-slave-vk40"
server_key: "eb86637ad34b3523ec91ff89296e8723"
server_key_iv: "3d4a515ade0031a464e6bb3603028e61"
server_key_version: "encryptionkey@0"
I20260928 14:55:13.185803 2170 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260928 14:55:13.189781 2185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:13.191202 2170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:55:13.191390 2170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "5a5da690d2ba4bad971e31ec8c3aa325"
format_stamp: "Formatted at 2026-09-28 14:55:13 on dist-test-slave-vk40"
server_key: "eb86637ad34b3523ec91ff89296e8723"
server_key_iv: "3d4a515ade0031a464e6bb3603028e61"
server_key_version: "encryptionkey@0"
I20260928 14:55:13.191596 2170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:13.215442 2170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:13.219755 2170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:13.220026 2170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:13.237795 2170 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:34703
I20260928 14:55:13.237808 2237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:34703 every 8 connection(s)
I20260928 14:55:13.239183 2170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:13.243053 2238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:13.246227 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 2170
I20260928 14:55:13.246408 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:13.246934 15303 external_mini_cluster.cc:1428] Setting key c1ac4950f9611f09c6bbd5a30344ad09
I20260928 14:55:13.255681 2238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325: Bootstrap starting.
I20260928 14:55:13.259653 2238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:13.261020 2238 log.cc:824] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325: Log is configured to *not* fsync() on all Append() calls
Sep 28 14:55:13 dist-test-slave-vk40 krb5kdc[2157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:13.264096 2238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325: No bootstrap required, opened a new log
I20260928 14:55:13.270363 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:13.248938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.14.241.254:34703)
0928 14:55:13.249712 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:13.249725 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:13.249768 (+ 43us) server_negotiation.cc:407] Connection header received
0928 14:55:13.251495 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:13.251520 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:13.251974 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:13.252360 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:13.254136 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:13.255531 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:13.257367 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:13.258082 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:13.258209 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:13.263028 (+ 4819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:13.263036 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:13.263052 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:13.263093 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:13.266087 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:13.266935 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:13.266943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:13.266953 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:13.267124 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:13.267509 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:13.267515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:13.267517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:13.268093 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:13.268428 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:13.268898 (+ 470us) server_negotiation.cc:299] Negotiation successful
0928 14:55:13.269332 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":356,"threads_started":1}
I20260928 14:55:13.272228 2238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a5da690d2ba4bad971e31ec8c3aa325" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34703 } }
I20260928 14:55:13.272738 2238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:13.272841 2238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a5da690d2ba4bad971e31ec8c3aa325, State: Initialized, Role: FOLLOWER
I20260928 14:55:13.273475 2238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a5da690d2ba4bad971e31ec8c3aa325" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34703 } }
I20260928 14:55:13.273744 2238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:13.273847 2238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:13.273978 2238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:13.276065 2238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a5da690d2ba4bad971e31ec8c3aa325" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34703 } }
I20260928 14:55:13.276587 2238 leader_election.cc:302] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a5da690d2ba4bad971e31ec8c3aa325; no voters:
I20260928 14:55:13.277103 2238 leader_election.cc:288] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:13.277230 2243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:13.277834 2243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [term 1 LEADER]: Becoming Leader. State: Replica: 5a5da690d2ba4bad971e31ec8c3aa325, State: Running, Role: LEADER
I20260928 14:55:13.278369 2243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a5da690d2ba4bad971e31ec8c3aa325" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34703 } }
I20260928 14:55:13.279189 2238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:13.281177 2245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a5da690d2ba4bad971e31ec8c3aa325. Latest consensus state: current_term: 1 leader_uuid: "5a5da690d2ba4bad971e31ec8c3aa325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a5da690d2ba4bad971e31ec8c3aa325" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34703 } } }
I20260928 14:55:13.281201 2244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a5da690d2ba4bad971e31ec8c3aa325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a5da690d2ba4bad971e31ec8c3aa325" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34703 } } }
I20260928 14:55:13.281602 2245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:13.281607 2244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:13.283469 2248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:13.288573 2248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:13.296833 2248 catalog_manager.cc:1347] Generated new cluster ID: abe3ffd9848740538ff4d39fc0eb0108
I20260928 14:55:13.296950 2248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:13.312996 2248 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:13.314925 2248 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:13.329232 2248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325: Generated new TSK 0
I20260928 14:55:13.330289 2248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
Entry for principal kudu/127.14.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab.
I20260928 14:55:13.399118 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34703
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:13.722213 2264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:13.722498 2264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:13.722591 2264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:13.729933 2264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:13.730057 2264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:13.730172 2264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:55:13.739933 2264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.193.keytab
--principal=kudu/127.14.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:13.741266 2264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:13.742486 2264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:13.755270 2269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:13.755230 2270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:13.755298 2272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:13.756147 2264 server_base.cc:1034] running on GCE node
I20260928 14:55:13.756770 2264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:13.757660 2264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:13.758857 2264 hybrid_clock.cc:648] HybridClock initialized: now 1790607313758795 us; error 49 us; skew 500 ppm
I20260928 14:55:13.759142 2264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:13 dist-test-slave-vk40 krb5kdc[2157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607313, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:13.762072 2264 init.cc:377] Logged in from keytab as kudu/127.14.241.193@KRBTEST.COM (short username kudu)
I20260928 14:55:13.763530 2264 webserver.cc:460] Webserver started at http://127.14.241.193:37851/ using document root <none> and password file <none>
I20260928 14:55:13.764192 2264 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:13.764336 2264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:13.764642 2264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:13.767494 2264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "076d8758143d4f9f856ce8b6c0885725"
format_stamp: "Formatted at 2026-09-28 14:55:13 on dist-test-slave-vk40"
server_key: "03a82fe1f826279d86c3515c0f05b1b6"
server_key_iv: "e408bd5cd366ca079a50f358ffe648e5"
server_key_version: "encryptionkey@0"
I20260928 14:55:13.768224 2264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "076d8758143d4f9f856ce8b6c0885725"
format_stamp: "Formatted at 2026-09-28 14:55:13 on dist-test-slave-vk40"
server_key: "03a82fe1f826279d86c3515c0f05b1b6"
server_key_iv: "e408bd5cd366ca079a50f358ffe648e5"
server_key_version: "encryptionkey@0"
I20260928 14:55:13.773277 2264 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260928 14:55:13.776666 2279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:13.777925 2264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260928 14:55:13.778069 2264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "076d8758143d4f9f856ce8b6c0885725"
format_stamp: "Formatted at 2026-09-28 14:55:13 on dist-test-slave-vk40"
server_key: "03a82fe1f826279d86c3515c0f05b1b6"
server_key_iv: "e408bd5cd366ca079a50f358ffe648e5"
server_key_version: "encryptionkey@0"
I20260928 14:55:13.778241 2264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:13.793689 2264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:13.797657 2264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:13.797925 2264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:13.798838 2264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:13.800127 2264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:13.800200 2264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:13.800336 2264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:13.800398 2264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:13.829981 2264 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:33709
I20260928 14:55:13.830030 2392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:33709 every 8 connection(s)
I20260928 14:55:13.831218 2264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:55:13.832312 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 2264
I20260928 14:55:13.832499 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:55:13.833060 15303 external_mini_cluster.cc:1428] Setting key 298205cbd20c0db7ace97b76252f9b9c
Sep 28 14:55:13 dist-test-slave-vk40 krb5kdc[2157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607313, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.193@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:13.853643 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:13.833999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:43683 (local address 127.14.241.254:34703)
0928 14:55:13.835145 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:13.835149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:13.835869 (+ 720us) server_negotiation.cc:407] Connection header received
0928 14:55:13.838354 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:13.838358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:13.838433 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:13.838520 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:13.840471 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:13.841194 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:13.843295 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:13.843685 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:13.843773 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:13.847710 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:13.847717 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:13.847722 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:13.847761 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:13.850603 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:13.851336 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:13.851341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:13.851344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:13.851480 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:13.851868 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:13.851873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:13.851875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:13.852189 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:13.852370 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:13.853071 (+ 701us) server_negotiation.cc:299] Negotiation successful
0928 14:55:13.853308 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1045,"thread_start_us":953,"threads_started":1}
I20260928 14:55:13.854640 2396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:13.834550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:34703 (local address 127.14.241.193:43683)
0928 14:55:13.835683 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:13.835744 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:13.838064 (+ 2320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:13.838799 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:13.838811 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:13.839514 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:13.840312 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:13.840337 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:13.841354 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:13.841359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:13.843167 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:13.843181 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:13.843908 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:13.843914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:13.844195 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:13.845027 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:13.845056 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:13.847472 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:13.850748 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:13.850756 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:13.850770 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:13.851207 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:13.851565 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:13.851568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:13.851570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:13.851761 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:13.852321 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:13.852330 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:13.852685 (+ 355us) client_negotiation.cc:769] Sending connection context
0928 14:55:13.852922 (+ 237us) client_negotiation.cc:240] Negotiation successful
0928 14:55:13.853342 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"thread_start_us":634,"threads_started":1}
I20260928 14:55:13.856240 2393 heartbeater.cc:347] Connected to a master server at 127.14.241.254:34703
I20260928 14:55:13.856573 2393 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:13.857379 2393 heartbeater.cc:511] Master 127.14.241.254:34703 requested a full tablet report, sending...
I20260928 14:55:13.859483 2202 ts_manager.cc:194] Registered new tserver with Master: 076d8758143d4f9f856ce8b6c0885725 (127.14.241.193:33709)
WARNING: no policy specified for kudu/127.14.241.194@KRBTEST.COM; defaulting to no policy
I20260928 14:55:13.861030 2202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.193@KRBTEST.COM'} at 127.14.241.193:43683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
Entry for principal kudu/127.14.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab.
I20260928 14:55:13.889894 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34703
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:14.207218 2399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:14.207538 2399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:14.207679 2399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:14.214912 2399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:14.215042 2399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:14.215200 2399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:55:14.224943 2399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.194.keytab
--principal=kudu/127.14.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:14.226204 2399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:14.227490 2399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:14.240137 2404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:14.240257 2405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:14.240227 2407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:14.241559 2399 server_base.cc:1034] running on GCE node
I20260928 14:55:14.242192 2399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:14.243142 2399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:14.244383 2399 hybrid_clock.cc:648] HybridClock initialized: now 1790607314244312 us; error 67 us; skew 500 ppm
I20260928 14:55:14.244693 2399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:14 dist-test-slave-vk40 krb5kdc[2157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607314, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:14.247732 2399 init.cc:377] Logged in from keytab as kudu/127.14.241.194@KRBTEST.COM (short username kudu)
I20260928 14:55:14.249212 2399 webserver.cc:460] Webserver started at http://127.14.241.194:40549/ using document root <none> and password file <none>
I20260928 14:55:14.249864 2399 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:14.249977 2399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:14.250253 2399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:14.253088 2399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "4efdcefc2f1c4f31aff7338098de3a08"
format_stamp: "Formatted at 2026-09-28 14:55:14 on dist-test-slave-vk40"
server_key: "5397d592b18bacebe52f7e33b4d40363"
server_key_iv: "2a74ae2525676f8d7b1129ce82b2d1b4"
server_key_version: "encryptionkey@0"
I20260928 14:55:14.253820 2399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "4efdcefc2f1c4f31aff7338098de3a08"
format_stamp: "Formatted at 2026-09-28 14:55:14 on dist-test-slave-vk40"
server_key: "5397d592b18bacebe52f7e33b4d40363"
server_key_iv: "2a74ae2525676f8d7b1129ce82b2d1b4"
server_key_version: "encryptionkey@0"
I20260928 14:55:14.258970 2399 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:14.262287 2414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:14.263592 2399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:55:14.263751 2399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "4efdcefc2f1c4f31aff7338098de3a08"
format_stamp: "Formatted at 2026-09-28 14:55:14 on dist-test-slave-vk40"
server_key: "5397d592b18bacebe52f7e33b4d40363"
server_key_iv: "2a74ae2525676f8d7b1129ce82b2d1b4"
server_key_version: "encryptionkey@0"
I20260928 14:55:14.263932 2399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:14.276468 2399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:14.280258 2399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:14.280562 2399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:14.281474 2399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:14.282775 2399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:14.282842 2399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:14.282948 2399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:14.282997 2399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:14.312420 2399 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:38561
I20260928 14:55:14.312489 2527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:38561 every 8 connection(s)
I20260928 14:55:14.313771 2399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:55:14.315418 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 2399
I20260928 14:55:14.315591 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:55:14.316097 15303 external_mini_cluster.cc:1428] Setting key 79bdffb89ba186c1cf0554199efe2949
Sep 28 14:55:14 dist-test-slave-vk40 krb5kdc[2157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607314, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.194@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:14.333748 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:14.316425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:57947 (local address 127.14.241.254:34703)
0928 14:55:14.316635 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:14.316641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:14.317897 (+ 1256us) server_negotiation.cc:407] Connection header received
0928 14:55:14.319755 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:14.319760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:14.319843 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:14.319926 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:14.321491 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.322205 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.323730 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.324130 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.324239 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:14.328138 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:14.328143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:14.328146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:14.328179 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:14.330719 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:14.331570 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:14.331574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:14.331576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:14.331688 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:14.332108 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:14.332111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:14.332113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:14.332407 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:14.332514 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:14.333316 (+ 802us) server_negotiation.cc:299] Negotiation successful
0928 14:55:14.333510 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260928 14:55:14.335135 2530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:14.316984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:34703 (local address 127.14.241.194:57947)
0928 14:55:14.317716 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:14.317765 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:14.319498 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:14.320164 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:14.320176 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:14.320765 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:14.321332 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:14.321351 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.322346 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.322352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:14.323595 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:14.323607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.324359 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.324367 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:14.324591 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:14.325254 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:14.325283 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:14.327907 (+ 2624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:14.330870 (+ 2963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:14.330879 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:14.330897 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:14.331471 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:14.331782 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:14.331787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:14.331790 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:14.332023 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:14.332516 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:14.332524 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:14.332947 (+ 423us) client_negotiation.cc:769] Sending connection context
0928 14:55:14.333233 (+ 286us) client_negotiation.cc:240] Negotiation successful
0928 14:55:14.333788 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":301,"threads_started":1}
I20260928 14:55:14.336580 2528 heartbeater.cc:347] Connected to a master server at 127.14.241.254:34703
I20260928 14:55:14.336927 2528 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:14.337833 2528 heartbeater.cc:511] Master 127.14.241.254:34703 requested a full tablet report, sending...
I20260928 14:55:14.339610 2202 ts_manager.cc:194] Registered new tserver with Master: 4efdcefc2f1c4f31aff7338098de3a08 (127.14.241.194:38561)
I20260928 14:55:14.340801 2202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.194@KRBTEST.COM'} at 127.14.241.194:57947
WARNING: no policy specified for kudu/127.14.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
Entry for principal kudu/127.14.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab.
I20260928 14:55:14.371263 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34703
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:14.693239 2533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:14.693535 2533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:14.693666 2533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:14.700906 2533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:14.701031 2533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:14.701181 2533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:55:14.710964 2533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.195.keytab
--principal=kudu/127.14.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:14.712214 2533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:14.713564 2533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:14.725818 2538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:14.725755 2541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:14.725796 2539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:14.726758 2533 server_base.cc:1034] running on GCE node
I20260928 14:55:14.727367 2533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:14.728331 2533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:14.729534 2533 hybrid_clock.cc:648] HybridClock initialized: now 1790607314729484 us; error 47 us; skew 500 ppm
I20260928 14:55:14.729823 2533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:14 dist-test-slave-vk40 krb5kdc[2157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607314, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:14.732726 2533 init.cc:377] Logged in from keytab as kudu/127.14.241.195@KRBTEST.COM (short username kudu)
I20260928 14:55:14.734050 2533 webserver.cc:460] Webserver started at http://127.14.241.195:43581/ using document root <none> and password file <none>
I20260928 14:55:14.734686 2533 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:14.734797 2533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:14.735070 2533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:14.737879 2533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "0519806a287f48c7b43aee3638692162"
format_stamp: "Formatted at 2026-09-28 14:55:14 on dist-test-slave-vk40"
server_key: "5bd779c5d582e3c762c9d607ac801b3f"
server_key_iv: "65cd1a5fb3a50a35f339f6bbc8b4c76f"
server_key_version: "encryptionkey@0"
I20260928 14:55:14.738600 2533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "0519806a287f48c7b43aee3638692162"
format_stamp: "Formatted at 2026-09-28 14:55:14 on dist-test-slave-vk40"
server_key: "5bd779c5d582e3c762c9d607ac801b3f"
server_key_iv: "65cd1a5fb3a50a35f339f6bbc8b4c76f"
server_key_version: "encryptionkey@0"
I20260928 14:55:14.743521 2533 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 14:55:14.746836 2548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:14.748054 2533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:55:14.748198 2533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "0519806a287f48c7b43aee3638692162"
format_stamp: "Formatted at 2026-09-28 14:55:14 on dist-test-slave-vk40"
server_key: "5bd779c5d582e3c762c9d607ac801b3f"
server_key_iv: "65cd1a5fb3a50a35f339f6bbc8b4c76f"
server_key_version: "encryptionkey@0"
I20260928 14:55:14.748417 2533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:14.774252 2533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:14.778426 2533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:14.778682 2533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:14.779634 2533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:14.780970 2533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:14.781045 2533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:14.781159 2533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:14.781209 2533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:14.810590 2533 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:35237
I20260928 14:55:14.810632 2661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:35237 every 8 connection(s)
I20260928 14:55:14.811800 2533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:55:14.815303 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 2533
I20260928 14:55:14.815482 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:55:14.816030 15303 external_mini_cluster.cc:1428] Setting key 71fd53efffa8c9ed48e3fc2d86aa3115
Sep 28 14:55:14 dist-test-slave-vk40 krb5kdc[2157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607314, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.195@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:14.830955 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:14.814209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:39751 (local address 127.14.241.254:34703)
0928 14:55:14.814388 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:14.814393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:14.815621 (+ 1228us) server_negotiation.cc:407] Connection header received
0928 14:55:14.817751 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:14.817756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:14.817860 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:14.817942 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:14.819559 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.820367 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.821808 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.822216 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.822303 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:14.825424 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:14.825429 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:14.825432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:14.825465 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:14.827759 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:14.828461 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:14.828465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:14.828467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:14.828588 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:14.828998 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:14.829002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:14.829005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:14.829349 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:14.829481 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:14.830385 (+ 904us) server_negotiation.cc:299] Negotiation successful
0928 14:55:14.830619 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:55:14.831941 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:14.814653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:34703 (local address 127.14.241.195:39751)
0928 14:55:14.815452 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:14.815500 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:14.817499 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:14.818198 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:14.818210 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:14.818859 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:14.819415 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:14.819432 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.820476 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.820479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:14.821688 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:14.821704 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.822383 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.822388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:14.822575 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:14.823116 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:14.823137 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:14.825240 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:14.827890 (+ 2650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:14.827897 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:14.827909 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:14.828372 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:14.828706 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:14.828709 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:14.828712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:14.828892 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:14.829468 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:14.829475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:14.829942 (+ 467us) client_negotiation.cc:769] Sending connection context
0928 14:55:14.830243 (+ 301us) client_negotiation.cc:240] Negotiation successful
0928 14:55:14.830747 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":281,"threads_started":1}
I20260928 14:55:14.833209 2662 heartbeater.cc:347] Connected to a master server at 127.14.241.254:34703
I20260928 14:55:14.833505 2662 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:14.834299 2662 heartbeater.cc:511] Master 127.14.241.254:34703 requested a full tablet report, sending...
I20260928 14:55:14.835749 2202 ts_manager.cc:194] Registered new tserver with Master: 0519806a287f48c7b43aee3638692162 (127.14.241.195:35237)
I20260928 14:55:14.836562 2202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.241.195@KRBTEST.COM'} at 127.14.241.195:39751
I20260928 14:55:14.843376 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:55:14.859072 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:14.847209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.14.241.254:34703)
0928 14:55:14.847431 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:14.847438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:14.847730 (+ 292us) server_negotiation.cc:407] Connection header received
0928 14:55:14.847868 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:14.847872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:14.847934 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:14.848003 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:14.849371 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.850093 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.851396 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.851824 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.851924 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:14.854127 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:14.854133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:14.854136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:14.854172 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:14.856747 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:14.857382 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:14.857386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:14.857388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:14.857519 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:14.857852 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:14.857856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:14.857858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:14.858193 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:14.858398 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:14.858628 (+ 230us) server_negotiation.cc:299] Negotiation successful
0928 14:55:14.858788 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 14:55:14.864264 2393 heartbeater.cc:503] Master 127.14.241.254:34703 was elected leader, sending a full tablet report...
I20260928 14:55:14.862632 2202 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:55:14.869812 2202 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:55:14.902295 2675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:14.883928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33709 (local address 127.0.0.1:46690)
0928 14:55:14.894301 (+ 10373us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:14.894320 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:14.894442 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:14.895188 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:14.895192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:14.895222 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:14.895736 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:14.895745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.897103 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.897109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:14.899542 (+ 2433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:14.899555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.901206 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.901213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:14.901416 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:14.901475 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 14:55:14.901615 (+ 140us) client_negotiation.cc:240] Negotiation successful
0928 14:55:14.901810 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10245,"mutex_wait_us":323,"thread_start_us":9413,"threads_started":1}
I20260928 14:55:14.902361 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:14.881857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:35237 (local address 127.0.0.1:47646)
0928 14:55:14.893459 (+ 11602us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:14.893478 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:14.893619 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:14.894661 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:14.894668 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:14.894699 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:14.895110 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:14.895127 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.897139 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.897144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:14.900044 (+ 2900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:14.900059 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.901543 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.901549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:14.901707 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:14.901742 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:55:14.901821 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 14:55:14.901954 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11452,"mutex_wait_us":244,"thread_start_us":719,"threads_started":1}
I20260928 14:55:14.902868 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:14.881635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54538 (local address 127.14.241.194:38561)
0928 14:55:14.894604 (+ 12969us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:14.894612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:14.894635 (+ 23us) server_negotiation.cc:407] Connection header received
0928 14:55:14.894716 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:14.894721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:14.894896 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:14.895063 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:14.895849 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.897286 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.900207 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.901576 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.901728 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:14.901890 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:14.902105 (+ 215us) server_negotiation.cc:299] Negotiation successful
0928 14:55:14.902277 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12818,"thread_start_us":12712,"threads_started":1}
I20260928 14:55:14.903028 2677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:14.882480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47646 (local address 127.14.241.195:35237)
0928 14:55:14.883851 (+ 1371us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:14.883872 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:14.893585 (+ 9713us) server_negotiation.cc:407] Connection header received
0928 14:55:14.893796 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:14.893807 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:14.894129 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:14.894344 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:14.895338 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.897018 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.900234 (+ 3216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.901235 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.901413 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:14.901578 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:14.902547 (+ 969us) server_negotiation.cc:299] Negotiation successful
0928 14:55:14.902709 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1206,"thread_start_us":1062,"threads_started":1}
I20260928 14:55:14.903081 2679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:14.884398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46690 (local address 127.14.241.193:33709)
0928 14:55:14.893315 (+ 8917us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:14.893325 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:14.894601 (+ 1276us) server_negotiation.cc:407] Connection header received
0928 14:55:14.894702 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:14.894709 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:14.894969 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:14.895302 (+ 333us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:14.895889 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.896976 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.899698 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.900994 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.901190 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:14.901380 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:14.902595 (+ 1215us) server_negotiation.cc:299] Negotiation successful
0928 14:55:14.902760 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8712,"thread_start_us":9195,"threads_started":1}
I20260928 14:55:14.903684 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:14.881633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:38561 (local address 127.0.0.1:54538)
0928 14:55:14.883011 (+ 1378us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:14.883061 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:14.883242 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:14.895083 (+ 11841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:14.895088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:14.895112 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:14.895647 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:14.895661 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.897425 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.897430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:14.900047 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:14.900059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:14.901766 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:14.901772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:14.901925 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:14.901960 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:55:14.902053 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 14:55:14.902173 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"mutex_wait_us":120,"thread_start_us":12377,"threads_started":1}
I20260928 14:55:14.906677 2327 tablet_service.cc:1467] Processing CreateTablet for tablet 5eb6b78bbf9c49cf895eeb52444d1664 (DEFAULT_TABLE table=test-table [id=c63a424b148240699f5b7ea7ac57edec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:14.906926 2596 tablet_service.cc:1467] Processing CreateTablet for tablet 5eb6b78bbf9c49cf895eeb52444d1664 (DEFAULT_TABLE table=test-table [id=c63a424b148240699f5b7ea7ac57edec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:14.906925 2462 tablet_service.cc:1467] Processing CreateTablet for tablet 5eb6b78bbf9c49cf895eeb52444d1664 (DEFAULT_TABLE table=test-table [id=c63a424b148240699f5b7ea7ac57edec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:14.907928 2327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eb6b78bbf9c49cf895eeb52444d1664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:14.908255 2462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eb6b78bbf9c49cf895eeb52444d1664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:14.908253 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eb6b78bbf9c49cf895eeb52444d1664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:14.918642 2680 tablet_bootstrap.cc:492] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725: Bootstrap starting.
I20260928 14:55:14.922348 2681 tablet_bootstrap.cc:492] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: Bootstrap starting.
I20260928 14:55:14.923166 2682 tablet_bootstrap.cc:492] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: Bootstrap starting.
I20260928 14:55:14.923278 2680 tablet_bootstrap.cc:654] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:14.924595 2680 log.cc:824] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:14.926700 2681 tablet_bootstrap.cc:654] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:14.927048 2682 tablet_bootstrap.cc:654] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:14.927870 2680 tablet_bootstrap.cc:492] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725: No bootstrap required, opened a new log
I20260928 14:55:14.927875 2681 log.cc:824] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:14.927933 2682 log.cc:824] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:14.928143 2680 ts_tablet_manager.cc:1397] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260928 14:55:14.930816 2681 tablet_bootstrap.cc:492] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: No bootstrap required, opened a new log
I20260928 14:55:14.930815 2682 tablet_bootstrap.cc:492] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: No bootstrap required, opened a new log
I20260928 14:55:14.931100 2681 ts_tablet_manager.cc:1397] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260928 14:55:14.931142 2682 ts_tablet_manager.cc:1397] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260928 14:55:14.936335 2680 raft_consensus.cc:348] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4efdcefc2f1c4f31aff7338098de3a08" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 38561 } } peers { permanent_uuid: "0519806a287f48c7b43aee3638692162" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35237 } } peers { permanent_uuid: "076d8758143d4f9f856ce8b6c0885725" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33709 } }
I20260928 14:55:14.936705 2680 raft_consensus.cc:374] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:14.936793 2680 raft_consensus.cc:729] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076d8758143d4f9f856ce8b6c0885725, State: Initialized, Role: FOLLOWER
I20260928 14:55:14.937384 2680 consensus_queue.cc:260] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4efdcefc2f1c4f31aff7338098de3a08" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 38561 } } peers { permanent_uuid: "0519806a287f48c7b43aee3638692162" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35237 } } peers { permanent_uuid: "076d8758143d4f9f856ce8b6c0885725" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33709 } }
I20260928 14:55:14.939126 2680 ts_tablet_manager.cc:1428] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260928 14:55:14.941322 2681 raft_consensus.cc:348] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4efdcefc2f1c4f31aff7338098de3a08" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 38561 } } peers { permanent_uuid: "0519806a287f48c7b43aee3638692162" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35237 } } peers { permanent_uuid: "076d8758143d4f9f856ce8b6c0885725" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33709 } }
I20260928 14:55:14.941673 2681 raft_consensus.cc:374] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:14.941767 2681 raft_consensus.cc:729] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0519806a287f48c7b43aee3638692162, State: Initialized, Role: FOLLOWER
I20260928 14:55:14.942106 2682 raft_consensus.cc:348] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4efdcefc2f1c4f31aff7338098de3a08" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 38561 } } peers { permanent_uuid: "0519806a287f48c7b43aee3638692162" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35237 } } peers { permanent_uuid: "076d8758143d4f9f856ce8b6c0885725" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33709 } }
I20260928 14:55:14.942355 2681 consensus_queue.cc:260] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4efdcefc2f1c4f31aff7338098de3a08" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 38561 } } peers { permanent_uuid: "0519806a287f48c7b43aee3638692162" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35237 } } peers { permanent_uuid: "076d8758143d4f9f856ce8b6c0885725" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33709 } }
I20260928 14:55:14.942538 2682 raft_consensus.cc:374] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:14.942643 2682 raft_consensus.cc:729] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4efdcefc2f1c4f31aff7338098de3a08, State: Initialized, Role: FOLLOWER
I20260928 14:55:14.943423 2662 heartbeater.cc:503] Master 127.14.241.254:34703 was elected leader, sending a full tablet report...
I20260928 14:55:14.943336 2682 consensus_queue.cc:260] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4efdcefc2f1c4f31aff7338098de3a08" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 38561 } } peers { permanent_uuid: "0519806a287f48c7b43aee3638692162" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35237 } } peers { permanent_uuid: "076d8758143d4f9f856ce8b6c0885725" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33709 } }
I20260928 14:55:14.944114 2681 ts_tablet_manager.cc:1428] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260928 14:55:14.946691 2528 heartbeater.cc:503] Master 127.14.241.254:34703 was elected leader, sending a full tablet report...
I20260928 14:55:14.947439 2682 ts_tablet_manager.cc:1428] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: Time spent starting tablet: real 0.016s user 0.008s sys 0.006s
W20260928 14:55:15.063336 2663 tablet.cc:2369] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:55:15.065632 2529 tablet.cc:2369] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:55:15.083483 2394 tablet.cc:2369] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:55:15.137149 2686 raft_consensus.cc:482] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:55:15.137462 2686 raft_consensus.cc:504] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4efdcefc2f1c4f31aff7338098de3a08" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 38561 } } peers { permanent_uuid: "0519806a287f48c7b43aee3638692162" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35237 } } peers { permanent_uuid: "076d8758143d4f9f856ce8b6c0885725" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33709 } }
I20260928 14:55:15.139204 2686 leader_election.cc:288] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4efdcefc2f1c4f31aff7338098de3a08 (127.14.241.194:38561), 0519806a287f48c7b43aee3638692162 (127.14.241.195:35237)
I20260928 14:55:15.147930 2677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.139747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:39461 (local address 127.14.241.195:35237)
0928 14:55:15.139921 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:15.139926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:15.140580 (+ 654us) server_negotiation.cc:407] Connection header received
0928 14:55:15.140806 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:15.140810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:15.140896 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:15.140974 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:15.141727 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.142826 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.145626 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.146846 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.146998 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:15.147072 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:15.147528 (+ 456us) server_negotiation.cc:299] Negotiation successful
0928 14:55:15.147660 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:55:15.147922 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.139661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:39877 (local address 127.14.241.194:38561)
0928 14:55:15.139881 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:15.139887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:15.140580 (+ 693us) server_negotiation.cc:407] Connection header received
0928 14:55:15.140774 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:15.140779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:15.140863 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:15.140967 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:15.141727 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.142826 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.145629 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.146846 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.146997 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:15.147073 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:15.147528 (+ 455us) server_negotiation.cc:299] Negotiation successful
0928 14:55:15.147660 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 14:55:15.148156 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.139534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:35237 (local address 127.14.241.193:39461)
0928 14:55:15.140450 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:15.140470 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:15.140670 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:15.140990 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:15.140995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:15.141030 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:15.141611 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:15.141621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.142948 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.142953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:15.145463 (+ 2510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:15.145477 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.147074 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.147081 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:15.147324 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:15.147397 (+ 73us) client_negotiation.cc:769] Sending connection context
0928 14:55:15.147492 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 14:55:15.147689 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":193,"spinlock_wait_cycles":37120,"thread_start_us":530,"threads_started":1}
I20260928 14:55:15.148499 2689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.139636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:38561 (local address 127.14.241.193:39877)
0928 14:55:15.140458 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:15.140471 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:15.140628 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:15.140990 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:15.140995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:15.141038 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:15.141537 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:15.141556 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.142956 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.142959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:15.145460 (+ 2501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:15.145477 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.147105 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.147110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:15.147343 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:15.147397 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 14:55:15.147492 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 14:55:15.147689 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"mutex_wait_us":226,"spinlock_wait_cycles":3968,"thread_start_us":431,"threads_started":1}
I20260928 14:55:15.149151 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eb6b78bbf9c49cf895eeb52444d1664" candidate_uuid: "076d8758143d4f9f856ce8b6c0885725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0519806a287f48c7b43aee3638692162" is_pre_election: true
I20260928 14:55:15.149313 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eb6b78bbf9c49cf895eeb52444d1664" candidate_uuid: "076d8758143d4f9f856ce8b6c0885725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4efdcefc2f1c4f31aff7338098de3a08" is_pre_election: true
I20260928 14:55:15.149650 2616 raft_consensus.cc:2454] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 076d8758143d4f9f856ce8b6c0885725 in term 0.
I20260928 14:55:15.149794 2482 raft_consensus.cc:2454] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 076d8758143d4f9f856ce8b6c0885725 in term 0.
I20260928 14:55:15.150329 2282 leader_election.cc:302] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0519806a287f48c7b43aee3638692162, 076d8758143d4f9f856ce8b6c0885725; no voters:
I20260928 14:55:15.150712 2686 raft_consensus.cc:2781] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:55:15.150859 2686 raft_consensus.cc:482] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:55:15.150969 2686 raft_consensus.cc:3037] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:15.153214 2686 raft_consensus.cc:504] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4efdcefc2f1c4f31aff7338098de3a08" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 38561 } } peers { permanent_uuid: "0519806a287f48c7b43aee3638692162" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35237 } } peers { permanent_uuid: "076d8758143d4f9f856ce8b6c0885725" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33709 } }
I20260928 14:55:15.153813 2686 leader_election.cc:288] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [CANDIDATE]: Term 1 election: Requested vote from peers 4efdcefc2f1c4f31aff7338098de3a08 (127.14.241.194:38561), 0519806a287f48c7b43aee3638692162 (127.14.241.195:35237)
I20260928 14:55:15.154300 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eb6b78bbf9c49cf895eeb52444d1664" candidate_uuid: "076d8758143d4f9f856ce8b6c0885725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4efdcefc2f1c4f31aff7338098de3a08"
I20260928 14:55:15.154300 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eb6b78bbf9c49cf895eeb52444d1664" candidate_uuid: "076d8758143d4f9f856ce8b6c0885725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0519806a287f48c7b43aee3638692162"
I20260928 14:55:15.154506 2482 raft_consensus.cc:3037] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:15.154505 2616 raft_consensus.cc:3037] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:15.157582 2616 raft_consensus.cc:2454] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 076d8758143d4f9f856ce8b6c0885725 in term 1.
I20260928 14:55:15.157581 2482 raft_consensus.cc:2454] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 076d8758143d4f9f856ce8b6c0885725 in term 1.
I20260928 14:55:15.157979 2282 leader_election.cc:302] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0519806a287f48c7b43aee3638692162, 076d8758143d4f9f856ce8b6c0885725, 4efdcefc2f1c4f31aff7338098de3a08; no voters:
I20260928 14:55:15.158278 2686 raft_consensus.cc:2781] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:15.158687 2686 raft_consensus.cc:686] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 1 LEADER]: Becoming Leader. State: Replica: 076d8758143d4f9f856ce8b6c0885725, State: Running, Role: LEADER
I20260928 14:55:15.159181 2686 consensus_queue.cc:237] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4efdcefc2f1c4f31aff7338098de3a08" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 38561 } } peers { permanent_uuid: "0519806a287f48c7b43aee3638692162" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35237 } } peers { permanent_uuid: "076d8758143d4f9f856ce8b6c0885725" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33709 } }
I20260928 14:55:15.163450 2202 catalog_manager.cc:5495] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 reported cstate change: term changed from 0 to 1, leader changed from <none> to 076d8758143d4f9f856ce8b6c0885725 (127.14.241.193). New cstate: current_term: 1 leader_uuid: "076d8758143d4f9f856ce8b6c0885725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4efdcefc2f1c4f31aff7338098de3a08" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 38561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0519806a287f48c7b43aee3638692162" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "076d8758143d4f9f856ce8b6c0885725" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 33709 } health_report { overall_health: HEALTHY } } }
I20260928 14:55:15.186990 2679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.181961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.14.241.193:33709)
0928 14:55:15.182152 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:15.182157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:15.182240 (+ 83us) server_negotiation.cc:407] Connection header received
0928 14:55:15.182419 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:15.182423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:15.182505 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:15.182626 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:55:15.182933 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.183678 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.185041 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.185494 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.185584 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:15.185997 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:55:15.186492 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:55:15.186598 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:15.186652 (+ 54us) server_negotiation.cc:299] Negotiation successful
0928 14:55:15.186721 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 14:55:15.195032 2482 raft_consensus.cc:1261] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08 [term 1 FOLLOWER]: Refusing update from remote peer 076d8758143d4f9f856ce8b6c0885725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:15.195029 2616 raft_consensus.cc:1261] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162 [term 1 FOLLOWER]: Refusing update from remote peer 076d8758143d4f9f856ce8b6c0885725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:15.196040 2691 consensus_queue.cc:1035] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4efdcefc2f1c4f31aff7338098de3a08" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 38561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:15.196358 2686 consensus_queue.cc:1035] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0519806a287f48c7b43aee3638692162" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 35237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:15.207309 2694 mvcc.cc:205] Tried to move back new op lower bound from 7334327563029061632 to 7334327562895392768. Current Snapshot: MvccSnapshot[applied={T|T < 7334327563029061632}]
I20260928 14:55:15.220734 2201 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:15.221069 2201 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:15.224939 2201 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a5da690d2ba4bad971e31ec8c3aa325: Sending DeleteTablet for 3 replicas of tablet 5eb6b78bbf9c49cf895eeb52444d1664
I20260928 14:55:15.226089 2462 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eb6b78bbf9c49cf895eeb52444d1664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:15 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:54538
I20260928 14:55:15.226143 2327 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eb6b78bbf9c49cf895eeb52444d1664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:15 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:46690
I20260928 14:55:15.226209 2596 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eb6b78bbf9c49cf895eeb52444d1664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:15 UTC) from {username='kudu', principal='kudu/127.14.241.254@KRBTEST.COM'} at 127.0.0.1:47646
I20260928 14:55:15.227793 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 2264
Sep 28 14:55:15 dist-test-slave-vk40 krb5kdc[2157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.193@KRBTEST.COM
I20260928 14:55:15.255977 2706 tablet_replica.cc:330] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725: stopping tablet replica
I20260928 14:55:15.258181 2706 raft_consensus.cc:2229] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:55:15.259346 2679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.228823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46698 (local address 127.14.241.193:33709)
0928 14:55:15.229004 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:15.229011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:15.236758 (+ 7747us) server_negotiation.cc:407] Connection header received
0928 14:55:15.236963 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:15.236969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:15.237063 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:15.237173 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:15.238964 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.240256 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.242297 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.243049 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.243145 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:15.248926 (+ 5781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:15.248944 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:15.248966 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:15.249022 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:15.253384 (+ 4362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:15.254689 (+ 1305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:15.254705 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:15.254709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:15.254897 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:15.256369 (+ 1472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:15.256375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:15.256379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:15.256902 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:15.257230 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:15.258636 (+ 1406us) server_negotiation.cc:299] Negotiation successful
0928 14:55:15.258924 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 14:55:15.261103 2388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:15.268337 2705 tablet_replica.cc:330] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: stopping tablet replica
I20260928 14:55:15.269076 2704 tablet_replica.cc:330] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: stopping tablet replica
I20260928 14:55:15.269413 2704 raft_consensus.cc:2229] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:15.269402 2705 raft_consensus.cc:2229] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:15.269943 2704 raft_consensus.cc:2258] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:15.270764 2705 raft_consensus.cc:2258] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:15.272600 2704 ts_tablet_manager.cc:1905] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:15.277846 2704 ts_tablet_manager.cc:1918] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:15.277997 2704 log.cc:1196] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/5eb6b78bbf9c49cf895eeb52444d1664
I20260928 14:55:15.278424 2704 ts_tablet_manager.cc:1939] T 5eb6b78bbf9c49cf895eeb52444d1664 P 4efdcefc2f1c4f31aff7338098de3a08: Deleting consensus metadata
I20260928 14:55:15.279992 2705 ts_tablet_manager.cc:1905] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:15.280004 2187 catalog_manager.cc:4841] TS 4efdcefc2f1c4f31aff7338098de3a08 (127.14.241.194:38561): tablet 5eb6b78bbf9c49cf895eeb52444d1664 (table test-table [id=c63a424b148240699f5b7ea7ac57edec]) successfully deleted
I20260928 14:55:15.287259 2705 ts_tablet_manager.cc:1918] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:15.287462 2705 log.cc:1196] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/5eb6b78bbf9c49cf895eeb52444d1664
I20260928 14:55:15.287957 2705 ts_tablet_manager.cc:1939] T 5eb6b78bbf9c49cf895eeb52444d1664 P 0519806a287f48c7b43aee3638692162: Deleting consensus metadata
I20260928 14:55:15.289997 2188 catalog_manager.cc:4841] TS 0519806a287f48c7b43aee3638692162 (127.14.241.195:35237): tablet 5eb6b78bbf9c49cf895eeb52444d1664 (table test-table [id=c63a424b148240699f5b7ea7ac57edec]) successfully deleted
I20260928 14:55:15.373759 2706 raft_consensus.cc:2258] T 5eb6b78bbf9c49cf895eeb52444d1664 P 076d8758143d4f9f856ce8b6c0885725 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:15.374250 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 2264
I20260928 14:55:15.390318 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 2399
W20260928 14:55:15.390357 2189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.241.193:33709 (error 108)
I20260928 14:55:15.392030 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.391514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33709 (local address 127.0.0.1:46714)
0928 14:55:15.391737 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:15.391860 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:33709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260928 14:55:15.392336 2189 catalog_manager.cc:4584] TS 076d8758143d4f9f856ce8b6c0885725 (127.14.241.193:33709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5eb6b78bbf9c49cf895eeb52444d1664: Network error: Client connection negotiation failed: client connection to 127.14.241.193:33709: connect: Connection refused (error 111)
Sep 28 14:55:15 dist-test-slave-vk40 krb5kdc[2157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.194@KRBTEST.COM
I20260928 14:55:15.406280 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.391036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54540 (local address 127.14.241.194:38561)
0928 14:55:15.391202 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:15.391207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:15.391416 (+ 209us) server_negotiation.cc:407] Connection header received
0928 14:55:15.391569 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:15.391574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:15.391672 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:15.391776 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:15.393609 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.394746 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.396462 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.396913 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.397014 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:15.400915 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:15.400930 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:15.400945 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:15.400988 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:15.403681 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:15.404527 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:15.404534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:15.404536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:15.404663 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:15.404969 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:15.404972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:15.404974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:15.405352 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:15.405583 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:15.405781 (+ 198us) server_negotiation.cc:299] Negotiation successful
0928 14:55:15.405962 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:55:15.406941 2523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:15.446219 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.445752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33709 (local address 127.0.0.1:46716)
0928 14:55:15.445963 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:15.446075 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:33709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260928 14:55:15.447216 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.446877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33709 (local address 127.0.0.1:46728)
0928 14:55:15.447021 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:15.447103 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:33709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260928 14:55:15.495378 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 2399
I20260928 14:55:15.508153 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.507700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33709 (local address 127.0.0.1:46738)
0928 14:55:15.507904 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:15.507993 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:33709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260928 14:55:15.509117 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.508774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33709 (local address 127.0.0.1:46754)
0928 14:55:15.508933 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:15.509001 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:33709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260928 14:55:15.509727 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 2533
Sep 28 14:55:15 dist-test-slave-vk40 krb5kdc[2157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.195@KRBTEST.COM
I20260928 14:55:15.524214 2677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.510380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47652 (local address 127.14.241.195:35237)
0928 14:55:15.510562 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:15.510568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:15.510609 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:55:15.510671 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:15.510675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:15.510735 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:15.510815 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:15.512086 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.512861 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.514182 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:15.514587 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:15.514698 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:15.518882 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:15.518898 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:15.518914 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:15.518958 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:15.521409 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:15.522169 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:15.522177 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:15.522179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:15.522330 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:15.522706 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:15.522710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:15.522712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:15.523082 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:15.523382 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:15.523665 (+ 283us) server_negotiation.cc:299] Negotiation successful
0928 14:55:15.523881 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 14:55:15.524963 2657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:15.589886 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.589330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33709 (local address 127.0.0.1:46756)
0928 14:55:15.589554 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:15.589657 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:33709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260928 14:55:15.591049 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:15.590624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:33709 (local address 127.0.0.1:46768)
0928 14:55:15.590835 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:15.590930 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.193:33709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260928 14:55:15.614061 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 2533
I20260928 14:55:15.627905 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 2170
I20260928 14:55:15.629017 2233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:15.739739 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 2170
2026-09-28T14:55:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4212 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-28T14:55:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:15Z Disabled control of system clock
I20260928 14:55:15.811810 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:34753
--webserver_interface=127.14.241.254
--webserver_port=0
--builtin_ntp_servers=127.14.241.212:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:34753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:55:16.141453 2716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:16.141757 2716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:16.141846 2716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:16.149236 2716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:16.149364 2716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:16.149405 2716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:16.149461 2716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:16.149505 2716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:16.158953 2716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:34753
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:34753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:16.160197 2716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:16.161998 2716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:16.173594 2724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:16.173547 2722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:16.173547 2721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:16.174618 2716 server_base.cc:1034] running on GCE node
I20260928 14:55:16.175400 2716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:16.176841 2716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:16.178040 2716 hybrid_clock.cc:648] HybridClock initialized: now 1790607316177980 us; error 51 us; skew 500 ppm
I20260928 14:55:16.178375 2716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:16.180609 2716 webserver.cc:460] Webserver started at http://127.14.241.254:46235/ using document root <none> and password file <none>
I20260928 14:55:16.181306 2716 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:16.181412 2716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:16.181691 2716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:16.184767 2716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "c063f600a71f4271b6e2d00814df2588"
format_stamp: "Formatted at 2026-09-28 14:55:16 on dist-test-slave-vk40"
server_key: "3122f6e882888d594c893edc3d58edac"
server_key_iv: "b2e3de8ae321667d3bb7d7f24bb10f51"
server_key_version: "encryptionkey@0"
I20260928 14:55:16.185555 2716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "c063f600a71f4271b6e2d00814df2588"
format_stamp: "Formatted at 2026-09-28 14:55:16 on dist-test-slave-vk40"
server_key: "3122f6e882888d594c893edc3d58edac"
server_key_iv: "b2e3de8ae321667d3bb7d7f24bb10f51"
server_key_version: "encryptionkey@0"
I20260928 14:55:16.190721 2716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:16.194460 2730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:16.195920 2716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:55:16.196086 2716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "c063f600a71f4271b6e2d00814df2588"
format_stamp: "Formatted at 2026-09-28 14:55:16 on dist-test-slave-vk40"
server_key: "3122f6e882888d594c893edc3d58edac"
server_key_iv: "b2e3de8ae321667d3bb7d7f24bb10f51"
server_key_version: "encryptionkey@0"
I20260928 14:55:16.196274 2716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:16.225628 2716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:16.226471 2716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:16.226734 2716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:16.243960 2716 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:34753
I20260928 14:55:16.243985 2782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:34753 every 8 connection(s)
I20260928 14:55:16.245286 2716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:16.249011 2783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:16.254866 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 2716
I20260928 14:55:16.255041 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:16.255582 15303 external_mini_cluster.cc:1428] Setting key 1b08dcc2a8a2a77366a314f61772c786
I20260928 14:55:16.260731 2783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588: Bootstrap starting.
I20260928 14:55:16.263741 2783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:16.264743 2783 log.cc:824] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:16.267901 2783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588: No bootstrap required, opened a new log
I20260928 14:55:16.269102 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:16.257637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39802 (local address 127.14.241.254:34753)
0928 14:55:16.258688 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:16.258699 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:16.258746 (+ 47us) server_negotiation.cc:407] Connection header received
0928 14:55:16.260530 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:16.260553 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:16.261044 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:16.261406 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:16.261970 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:16.263282 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:16.264707 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:16.265658 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:16.265857 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:16.266894 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:16.266901 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:16.266916 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:16.267435 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:16.267513 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:16.267533 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:16.267616 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:16.267823 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 14:55:16.268126 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":358,"threads_started":1}
I20260928 14:55:16.275249 2783 raft_consensus.cc:348] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c063f600a71f4271b6e2d00814df2588" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34753 } }
I20260928 14:55:16.275602 2783 raft_consensus.cc:374] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:16.275694 2783 raft_consensus.cc:729] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c063f600a71f4271b6e2d00814df2588, State: Initialized, Role: FOLLOWER
I20260928 14:55:16.276221 2783 consensus_queue.cc:260] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c063f600a71f4271b6e2d00814df2588" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34753 } }
I20260928 14:55:16.276484 2783 raft_consensus.cc:388] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:16.276605 2783 raft_consensus.cc:482] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:16.276757 2783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:16.278889 2783 raft_consensus.cc:504] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c063f600a71f4271b6e2d00814df2588" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34753 } }
I20260928 14:55:16.279373 2783 leader_election.cc:302] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c063f600a71f4271b6e2d00814df2588; no voters:
I20260928 14:55:16.279913 2783 leader_election.cc:288] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:16.280028 2788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:16.280751 2788 raft_consensus.cc:686] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [term 1 LEADER]: Becoming Leader. State: Replica: c063f600a71f4271b6e2d00814df2588, State: Running, Role: LEADER
I20260928 14:55:16.281231 2788 consensus_queue.cc:237] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c063f600a71f4271b6e2d00814df2588" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34753 } }
I20260928 14:55:16.281944 2783 sys_catalog.cc:564] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:16.284054 2790 sys_catalog.cc:455] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c063f600a71f4271b6e2d00814df2588. Latest consensus state: current_term: 1 leader_uuid: "c063f600a71f4271b6e2d00814df2588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c063f600a71f4271b6e2d00814df2588" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34753 } } }
I20260928 14:55:16.284055 2789 sys_catalog.cc:455] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c063f600a71f4271b6e2d00814df2588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c063f600a71f4271b6e2d00814df2588" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34753 } } }
I20260928 14:55:16.284545 2789 sys_catalog.cc:458] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:16.284549 2790 sys_catalog.cc:458] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:16.286186 2793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:16.291530 2793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:16.302129 2793 catalog_manager.cc:1347] Generated new cluster ID: d6cd6b85df0f4e159dc82590d57ab314
I20260928 14:55:16.302255 2793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:16.322763 2793 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:16.324767 2793 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:16.346709 2793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588: Generated new TSK 0
I20260928 14:55:16.347872 2793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:16.354759 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.193:0
--local_ip_for_outbound_sockets=127.14.241.193
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34753
--builtin_ntp_servers=127.14.241.212:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:55:16.679972 2807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:16.680241 2807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:16.680377 2807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:16.687678 2807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:16.687774 2807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:16.687912 2807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.193
I20260928 14:55:16.697273 2807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.241.193
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.193
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:16.698513 2807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:16.700356 2807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:16.712960 2812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:16.712921 2815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:16.713040 2813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:16.713665 2807 server_base.cc:1034] running on GCE node
I20260928 14:55:16.714232 2807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:16.715106 2807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:16.716332 2807 hybrid_clock.cc:648] HybridClock initialized: now 1790607316716262 us; error 31 us; skew 500 ppm
I20260928 14:55:16.716630 2807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:16.718743 2807 webserver.cc:460] Webserver started at http://127.14.241.193:34631/ using document root <none> and password file <none>
I20260928 14:55:16.719394 2807 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:16.719496 2807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:16.719779 2807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:16.722682 2807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/data/instance:
uuid: "529958ee62a84527923ebf0f4c0c926d"
format_stamp: "Formatted at 2026-09-28 14:55:16 on dist-test-slave-vk40"
server_key: "82e056ac5c4a24e838ad4acfdb669120"
server_key_iv: "772e9767a4a49ec89fa08ab62e1f7fc3"
server_key_version: "encryptionkey@0"
I20260928 14:55:16.723469 2807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance:
uuid: "529958ee62a84527923ebf0f4c0c926d"
format_stamp: "Formatted at 2026-09-28 14:55:16 on dist-test-slave-vk40"
server_key: "82e056ac5c4a24e838ad4acfdb669120"
server_key_iv: "772e9767a4a49ec89fa08ab62e1f7fc3"
server_key_version: "encryptionkey@0"
I20260928 14:55:16.728596 2807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 14:55:16.732133 2821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:16.733466 2807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:55:16.733611 2807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/wal
uuid: "529958ee62a84527923ebf0f4c0c926d"
format_stamp: "Formatted at 2026-09-28 14:55:16 on dist-test-slave-vk40"
server_key: "82e056ac5c4a24e838ad4acfdb669120"
server_key_iv: "772e9767a4a49ec89fa08ab62e1f7fc3"
server_key_version: "encryptionkey@0"
I20260928 14:55:16.733804 2807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:16.760813 2807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:16.761663 2807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:16.761948 2807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:16.762914 2807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:16.764380 2807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:16.764474 2807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:16.764583 2807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:16.764652 2807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:16.794523 2807 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.193:39321
I20260928 14:55:16.794593 2933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.193:39321 every 8 connection(s)
I20260928 14:55:16.795900 2807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/data/info.pb
I20260928 14:55:16.799386 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 2807
I20260928 14:55:16.799551 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/wal/instance
I20260928 14:55:16.799978 15303 external_mini_cluster.cc:1428] Setting key a8ca7c8676600ec2128760e5f14cbb0a
I20260928 14:55:16.802930 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.194:0
--local_ip_for_outbound_sockets=127.14.241.194
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34753
--builtin_ntp_servers=127.14.241.212:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:55:16.808881 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:16.798323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.193:60945 (local address 127.14.241.254:34753)
0928 14:55:16.798732 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:16.798736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:16.800128 (+ 1392us) server_negotiation.cc:407] Connection header received
0928 14:55:16.802043 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:16.802047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:16.802111 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:16.802191 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:16.803575 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:16.804296 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:16.806268 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:16.806673 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:16.806761 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:16.808161 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:16.808166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:16.808169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:16.808186 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:16.808228 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:16.808239 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:16.808339 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:16.808544 (+ 205us) server_negotiation.cc:299] Negotiation successful
0928 14:55:16.808624 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":196,"threads_started":1}
I20260928 14:55:16.810134 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:16.798921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:34753 (local address 127.14.241.193:60945)
0928 14:55:16.799952 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:16.800004 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:16.801743 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:16.802464 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:16.802475 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:16.802593 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:16.803406 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:16.803432 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:16.804440 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:16.804446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:16.806150 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:16.806165 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:16.806878 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:16.806883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:16.807139 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:16.807817 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:55:16.807820 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:16.807894 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:55:16.807986 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:16.808350 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:16.808361 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:16.808369 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 14:55:16.808545 (+ 176us) client_negotiation.cc:240] Negotiation successful
0928 14:55:16.808934 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":312,"threads_started":1}
I20260928 14:55:16.811573 2934 heartbeater.cc:347] Connected to a master server at 127.14.241.254:34753
I20260928 14:55:16.811867 2934 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:16.812775 2934 heartbeater.cc:511] Master 127.14.241.254:34753 requested a full tablet report, sending...
I20260928 14:55:16.814884 2747 ts_manager.cc:194] Registered new tserver with Master: 529958ee62a84527923ebf0f4c0c926d (127.14.241.193:39321)
I20260928 14:55:16.816432 2747 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.193:60945
W20260928 14:55:17.133561 2938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:17.133860 2938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:17.133968 2938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:17.141443 2938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:17.141546 2938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:17.141688 2938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.194
I20260928 14:55:17.150990 2938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.241.194
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.194
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:17.152237 2938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:17.154030 2938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:17.166719 2943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:17.166693 2946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:17.166800 2944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:17.167901 2938 server_base.cc:1034] running on GCE node
I20260928 14:55:17.168565 2938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:17.169529 2938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:17.170729 2938 hybrid_clock.cc:648] HybridClock initialized: now 1790607317170673 us; error 58 us; skew 500 ppm
I20260928 14:55:17.171062 2938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:17.173482 2938 webserver.cc:460] Webserver started at http://127.14.241.194:35891/ using document root <none> and password file <none>
I20260928 14:55:17.174183 2938 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:17.174276 2938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:17.174575 2938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:17.177523 2938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/data/instance:
uuid: "8f786664c82f405387bf886abbd0cb8f"
format_stamp: "Formatted at 2026-09-28 14:55:17 on dist-test-slave-vk40"
server_key: "8b45a2618c5b9f2ff3d3a293ab28c50e"
server_key_iv: "aba43a985910608538cb4e23a3090793"
server_key_version: "encryptionkey@0"
I20260928 14:55:17.178313 2938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance:
uuid: "8f786664c82f405387bf886abbd0cb8f"
format_stamp: "Formatted at 2026-09-28 14:55:17 on dist-test-slave-vk40"
server_key: "8b45a2618c5b9f2ff3d3a293ab28c50e"
server_key_iv: "aba43a985910608538cb4e23a3090793"
server_key_version: "encryptionkey@0"
I20260928 14:55:17.183477 2938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260928 14:55:17.186945 2952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:17.188249 2938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:55:17.188462 2938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/wal
uuid: "8f786664c82f405387bf886abbd0cb8f"
format_stamp: "Formatted at 2026-09-28 14:55:17 on dist-test-slave-vk40"
server_key: "8b45a2618c5b9f2ff3d3a293ab28c50e"
server_key_iv: "aba43a985910608538cb4e23a3090793"
server_key_version: "encryptionkey@0"
I20260928 14:55:17.188663 2938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:17.214745 2938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:17.215595 2938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:17.215883 2938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:17.216997 2938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:17.218472 2938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:17.218567 2938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:17.218674 2938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:17.218744 2938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:17.250943 2938 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.194:46035
I20260928 14:55:17.250994 3065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.194:46035 every 8 connection(s)
I20260928 14:55:17.252235 2938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/data/info.pb
I20260928 14:55:17.259656 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 2938
I20260928 14:55:17.259850 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/wal/instance
I20260928 14:55:17.260430 15303 external_mini_cluster.cc:1428] Setting key a16f884ba671b505d9f988b98102ef24
I20260928 14:55:17.263314 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.241.195:0
--local_ip_for_outbound_sockets=127.14.241.195
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34753
--builtin_ntp_servers=127.14.241.212:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:55:17.265429 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:17.254832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.194:58433 (local address 127.14.241.254:34753)
0928 14:55:17.255039 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:17.255045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:17.256602 (+ 1557us) server_negotiation.cc:407] Connection header received
0928 14:55:17.258373 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:17.258380 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:17.258464 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:17.258587 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:17.259679 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.260760 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.262658 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.263245 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.263385 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:17.264689 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:17.264694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:17.264698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:17.264719 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:17.264759 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:17.264772 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:17.264859 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:17.265030 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 14:55:17.265128 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 14:55:17.266371 3068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:17.255363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:34753 (local address 127.14.241.194:58433)
0928 14:55:17.256427 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:17.256480 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:17.258112 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:17.258758 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:17.258765 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:17.258858 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:17.259459 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:17.259476 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.260935 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.260940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:17.262489 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:17.262501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.263455 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.263463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:17.263709 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:17.264382 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:55:17.264385 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:17.264425 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:55:17.264511 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:17.264855 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:17.264861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:17.264866 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:55:17.265003 (+ 137us) client_negotiation.cc:240] Negotiation successful
0928 14:55:17.265326 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":315,"threads_started":1}
I20260928 14:55:17.267735 3066 heartbeater.cc:347] Connected to a master server at 127.14.241.254:34753
I20260928 14:55:17.268028 3066 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:17.268818 3066 heartbeater.cc:511] Master 127.14.241.254:34753 requested a full tablet report, sending...
I20260928 14:55:17.270231 2747 ts_manager.cc:194] Registered new tserver with Master: 8f786664c82f405387bf886abbd0cb8f (127.14.241.194:46035)
I20260928 14:55:17.271047 2747 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.194:58433
W20260928 14:55:17.616158 3069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:17.616492 3069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:17.616621 3069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:17.623870 3069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:17.623978 3069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:17.624125 3069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.241.195
I20260928 14:55:17.633309 3069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.241.195
--webserver_port=0
--tserver_master_addrs=127.14.241.254:34753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.241.195
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:17.634586 3069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:17.636322 3069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:17.648963 3075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:17.648932 3077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:17.649013 3074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:17.649914 3069 server_base.cc:1034] running on GCE node
I20260928 14:55:17.650537 3069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:17.651471 3069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:17.652669 3069 hybrid_clock.cc:648] HybridClock initialized: now 1790607317652616 us; error 47 us; skew 500 ppm
I20260928 14:55:17.652987 3069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260928 14:55:17.655144 3069 webserver.cc:460] Webserver started at http://127.14.241.195:44119/ using document root <none> and password file <none>
I20260928 14:55:17.655827 3069 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:17.655946 3069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:17.656239 3069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:17.659111 3069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/data/instance:
uuid: "0e75c29fb04d42a69fcbcb0c45520ada"
format_stamp: "Formatted at 2026-09-28 14:55:17 on dist-test-slave-vk40"
server_key: "90fa236851e80f63718e086473e5cbbd"
server_key_iv: "157fa35e65b63dd6e5251ac01a8c1ac6"
server_key_version: "encryptionkey@0"
I20260928 14:55:17.659909 3069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance:
uuid: "0e75c29fb04d42a69fcbcb0c45520ada"
format_stamp: "Formatted at 2026-09-28 14:55:17 on dist-test-slave-vk40"
server_key: "90fa236851e80f63718e086473e5cbbd"
server_key_iv: "157fa35e65b63dd6e5251ac01a8c1ac6"
server_key_version: "encryptionkey@0"
I20260928 14:55:17.665028 3069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 14:55:17.668568 3083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:17.669962 3069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 14:55:17.670111 3069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/wal
uuid: "0e75c29fb04d42a69fcbcb0c45520ada"
format_stamp: "Formatted at 2026-09-28 14:55:17 on dist-test-slave-vk40"
server_key: "90fa236851e80f63718e086473e5cbbd"
server_key_iv: "157fa35e65b63dd6e5251ac01a8c1ac6"
server_key_version: "encryptionkey@0"
I20260928 14:55:17.670316 3069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:17.698964 3069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:17.699826 3069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:17.700096 3069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:17.701184 3069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:55:17.702626 3069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:55:17.702711 3069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:17.702816 3069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:55:17.702881 3069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:17.732517 3069 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.195:38415
I20260928 14:55:17.732569 3195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.195:38415 every 8 connection(s)
I20260928 14:55:17.733783 3069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/data/info.pb
I20260928 14:55:17.738575 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 3069
I20260928 14:55:17.738720 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/wal/instance
I20260928 14:55:17.739145 15303 external_mini_cluster.cc:1428] Setting key bad009427bc225495ba4224e59cfe197
I20260928 14:55:17.746227 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:17.736186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:34731 (local address 127.14.241.254:34753)
0928 14:55:17.736390 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:17.736395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:17.737828 (+ 1433us) server_negotiation.cc:407] Connection header received
0928 14:55:17.739827 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:17.739832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:17.739916 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:17.740020 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:17.741405 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.742290 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.743824 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.744234 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.744388 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:17.745587 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:17.745592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:17.745596 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:17.745614 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:17.745646 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:17.745657 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:17.745721 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:17.745886 (+ 165us) server_negotiation.cc:299] Negotiation successful
0928 14:55:17.745994 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:55:17.747288 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:17.736748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:34753 (local address 127.14.241.195:34731)
0928 14:55:17.737660 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:17.737704 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:17.739482 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:17.740308 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:17.740322 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:17.740452 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:17.741222 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:17.741248 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.742421 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.742425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:17.743686 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:17.743699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.744463 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.744468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:17.744678 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:17.745297 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:55:17.745299 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:17.745340 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:55:17.745423 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:17.745725 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:17.745732 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:17.745738 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:55:17.745885 (+ 147us) client_negotiation.cc:240] Negotiation successful
0928 14:55:17.746196 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":274,"threads_started":1}
I20260928 14:55:17.748656 3196 heartbeater.cc:347] Connected to a master server at 127.14.241.254:34753
I20260928 14:55:17.748936 3196 heartbeater.cc:464] Registering TS with master...
I20260928 14:55:17.749640 3196 heartbeater.cc:511] Master 127.14.241.254:34753 requested a full tablet report, sending...
I20260928 14:55:17.750978 2747 ts_manager.cc:194] Registered new tserver with Master: 0e75c29fb04d42a69fcbcb0c45520ada (127.14.241.195:38415)
I20260928 14:55:17.751794 2747 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.241.195:34731
I20260928 14:55:17.754357 15303 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:55:17.763765 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:17.758277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39806 (local address 127.14.241.254:34753)
0928 14:55:17.758426 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:17.758431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:17.758773 (+ 342us) server_negotiation.cc:407] Connection header received
0928 14:55:17.758909 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:17.758912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:17.758982 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:17.759058 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:17.759498 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.760183 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.761531 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.761992 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.762092 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:17.763076 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:17.763082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:17.763086 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:17.763108 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:17.763157 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:17.763171 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:17.763248 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:17.763424 (+ 176us) server_negotiation.cc:299] Negotiation successful
0928 14:55:17.763502 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:55:17.767571 2747 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:55:17.774385 2747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:55:17.812000 3213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:17.787918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54626 (local address 127.14.241.194:46035)
0928 14:55:17.803116 (+ 15198us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:17.803126 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:17.804008 (+ 882us) server_negotiation.cc:407] Connection header received
0928 14:55:17.804112 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:17.804120 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:17.804482 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:17.804677 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:17.805298 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.806762 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.808983 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.810387 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.810555 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:17.810790 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:17.811525 (+ 735us) server_negotiation.cc:299] Negotiation successful
0928 14:55:17.811700 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15019,"thread_start_us":15215,"threads_started":1}
I20260928 14:55:17.811990 3211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:17.786967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39321 (local address 127.0.0.1:33190)
0928 14:55:17.801379 (+ 14412us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:17.801403 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:17.801575 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:17.804325 (+ 2750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:17.804332 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:17.804363 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:17.804786 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:17.804806 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.806404 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.806409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:17.808991 (+ 2582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:17.809010 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.810993 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.810999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:17.811166 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:17.811254 (+ 88us) client_negotiation.cc:769] Sending connection context
0928 14:55:17.811378 (+ 124us) client_negotiation.cc:240] Negotiation successful
0928 14:55:17.811589 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14244,"mutex_wait_us":50,"thread_start_us":16509,"threads_started":1}
I20260928 14:55:17.812028 3208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:17.786142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.195:38415 (local address 127.0.0.1:53622)
0928 14:55:17.787197 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:17.787238 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:17.787424 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:17.805239 (+ 17815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:17.805244 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:17.805270 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:17.805605 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:17.805616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.807477 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.807483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:17.809490 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:17.809501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.811076 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.811083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:17.811295 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:17.811336 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 14:55:17.811413 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:55:17.811599 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":108,"thread_start_us":611,"threads_started":1}
I20260928 14:55:17.812084 3209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:17.786524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.14.241.195:38415)
0928 14:55:17.804667 (+ 18143us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:17.804674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:17.804705 (+ 31us) server_negotiation.cc:407] Connection header received
0928 14:55:17.804803 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:17.804810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:17.805042 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:17.805306 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:17.805779 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.807318 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.809655 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.810861 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.811329 (+ 468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:17.811551 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:17.811620 (+ 69us) server_negotiation.cc:299] Negotiation successful
0928 14:55:17.811775 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17985,"thread_start_us":17836,"threads_started":1}
I20260928 14:55:17.812683 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:17.787595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:46035 (local address 127.0.0.1:54626)
0928 14:55:17.803733 (+ 16138us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:17.803751 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:17.803880 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:17.804684 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:17.804689 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:17.804722 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:17.805089 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:17.805101 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.806943 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.806949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:17.808801 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:17.808814 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.810939 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.810946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:17.811179 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:17.811314 (+ 135us) client_negotiation.cc:769] Sending connection context
0928 14:55:17.811380 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 14:55:17.812461 (+ 1081us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16014,"mutex_wait_us":1021,"thread_start_us":14080,"threads_started":1}
I20260928 14:55:17.813946 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:17.787567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33190 (local address 127.14.241.193:39321)
0928 14:55:17.803391 (+ 15824us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:17.803400 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:17.803434 (+ 34us) server_negotiation.cc:407] Connection header received
0928 14:55:17.803540 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:17.803548 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:17.803831 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:17.804032 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:17.804974 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.806230 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.809182 (+ 2952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:17.810537 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:17.810709 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:17.810883 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:17.813445 (+ 2562us) server_negotiation.cc:299] Negotiation successful
0928 14:55:17.813615 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15661,"thread_start_us":15542,"threads_started":1}
I20260928 14:55:17.816311 3131 tablet_service.cc:1467] Processing CreateTablet for tablet a488a6d17d6c4f4c9260efefa827a042 (DEFAULT_TABLE table=test-table [id=9103956ca62f4455a992c94bf807325b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:17.816311 3001 tablet_service.cc:1467] Processing CreateTablet for tablet a488a6d17d6c4f4c9260efefa827a042 (DEFAULT_TABLE table=test-table [id=9103956ca62f4455a992c94bf807325b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:17.816876 2869 tablet_service.cc:1467] Processing CreateTablet for tablet a488a6d17d6c4f4c9260efefa827a042 (DEFAULT_TABLE table=test-table [id=9103956ca62f4455a992c94bf807325b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:55:17.817771 3131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a488a6d17d6c4f4c9260efefa827a042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:17.817767 3001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a488a6d17d6c4f4c9260efefa827a042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:17.818182 2869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a488a6d17d6c4f4c9260efefa827a042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:17.819308 2934 heartbeater.cc:503] Master 127.14.241.254:34753 was elected leader, sending a full tablet report...
I20260928 14:55:17.830159 3214 tablet_bootstrap.cc:492] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: Bootstrap starting.
I20260928 14:55:17.833069 3215 tablet_bootstrap.cc:492] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: Bootstrap starting.
I20260928 14:55:17.833890 3214 tablet_bootstrap.cc:654] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:17.835279 3214 log.cc:824] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:17.836671 3215 tablet_bootstrap.cc:654] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:17.837299 3216 tablet_bootstrap.cc:492] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: Bootstrap starting.
I20260928 14:55:17.838049 3215 log.cc:824] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:17.840674 3216 tablet_bootstrap.cc:654] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:17.841102 3215 tablet_bootstrap.cc:492] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: No bootstrap required, opened a new log
I20260928 14:55:17.841392 3215 ts_tablet_manager.cc:1397] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260928 14:55:17.841764 3214 tablet_bootstrap.cc:492] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: No bootstrap required, opened a new log
I20260928 14:55:17.841878 3216 log.cc:824] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:17.842027 3214 ts_tablet_manager.cc:1397] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.001s
I20260928 14:55:17.845908 3216 tablet_bootstrap.cc:492] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: No bootstrap required, opened a new log
I20260928 14:55:17.846169 3216 ts_tablet_manager.cc:1397] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260928 14:55:17.849468 3214 raft_consensus.cc:348] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38415 } } peers { permanent_uuid: "8f786664c82f405387bf886abbd0cb8f" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46035 } } peers { permanent_uuid: "529958ee62a84527923ebf0f4c0c926d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39321 } }
I20260928 14:55:17.849825 3214 raft_consensus.cc:374] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:17.849933 3214 raft_consensus.cc:729] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529958ee62a84527923ebf0f4c0c926d, State: Initialized, Role: FOLLOWER
I20260928 14:55:17.850503 3214 consensus_queue.cc:260] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38415 } } peers { permanent_uuid: "8f786664c82f405387bf886abbd0cb8f" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46035 } } peers { permanent_uuid: "529958ee62a84527923ebf0f4c0c926d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39321 } }
I20260928 14:55:17.851344 3215 raft_consensus.cc:348] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38415 } } peers { permanent_uuid: "8f786664c82f405387bf886abbd0cb8f" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46035 } } peers { permanent_uuid: "529958ee62a84527923ebf0f4c0c926d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39321 } }
I20260928 14:55:17.851784 3215 raft_consensus.cc:374] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:17.851886 3215 raft_consensus.cc:729] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f786664c82f405387bf886abbd0cb8f, State: Initialized, Role: FOLLOWER
I20260928 14:55:17.852095 3214 ts_tablet_manager.cc:1428] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260928 14:55:17.852484 3215 consensus_queue.cc:260] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38415 } } peers { permanent_uuid: "8f786664c82f405387bf886abbd0cb8f" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46035 } } peers { permanent_uuid: "529958ee62a84527923ebf0f4c0c926d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39321 } }
I20260928 14:55:17.853797 3066 heartbeater.cc:503] Master 127.14.241.254:34753 was elected leader, sending a full tablet report...
I20260928 14:55:17.854424 3215 ts_tablet_manager.cc:1428] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260928 14:55:17.857329 3216 raft_consensus.cc:348] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38415 } } peers { permanent_uuid: "8f786664c82f405387bf886abbd0cb8f" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46035 } } peers { permanent_uuid: "529958ee62a84527923ebf0f4c0c926d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39321 } }
I20260928 14:55:17.857788 3216 raft_consensus.cc:374] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:17.857878 3216 raft_consensus.cc:729] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e75c29fb04d42a69fcbcb0c45520ada, State: Initialized, Role: FOLLOWER
I20260928 14:55:17.858476 3216 consensus_queue.cc:260] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38415 } } peers { permanent_uuid: "8f786664c82f405387bf886abbd0cb8f" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46035 } } peers { permanent_uuid: "529958ee62a84527923ebf0f4c0c926d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39321 } }
I20260928 14:55:17.859473 3196 heartbeater.cc:503] Master 127.14.241.254:34753 was elected leader, sending a full tablet report...
I20260928 14:55:17.859867 3216 ts_tablet_manager.cc:1428] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260928 14:55:17.985339 3197 tablet.cc:2369] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:55:18.004320 3067 tablet.cc:2369] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:55:18.015223 3222 raft_consensus.cc:482] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:55:18.015527 3222 raft_consensus.cc:504] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38415 } } peers { permanent_uuid: "8f786664c82f405387bf886abbd0cb8f" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46035 } } peers { permanent_uuid: "529958ee62a84527923ebf0f4c0c926d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39321 } }
I20260928 14:55:18.017257 3222 leader_election.cc:288] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f786664c82f405387bf886abbd0cb8f (127.14.241.194:46035), 529958ee62a84527923ebf0f4c0c926d (127.14.241.193:39321)
I20260928 14:55:18.023005 3213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:18.017788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:38387 (local address 127.14.241.194:46035)
0928 14:55:18.018099 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:18.018105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:18.018126 (+ 21us) server_negotiation.cc:407] Connection header received
0928 14:55:18.018303 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:18.018307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:18.018390 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:18.018469 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:18.018960 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.019661 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.021380 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.022146 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.022288 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:18.022367 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:18.022610 (+ 243us) server_negotiation.cc:299] Negotiation successful
0928 14:55:18.022745 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260928 14:55:18.023038 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:18.017706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.194:46035 (local address 127.14.241.195:38387)
0928 14:55:18.017991 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:18.018012 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:18.018152 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:18.018560 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:18.018565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:18.018595 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:18.018793 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:18.018801 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.019793 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.019797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:18.021216 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:18.021229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.022293 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.022299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:18.022453 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:18.022477 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 14:55:18.022555 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:55:18.022692 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":2,"spinlock_wait_cycles":19712}
I20260928 14:55:18.023465 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:18.017745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.241.195:55781 (local address 127.14.241.193:39321)
0928 14:55:18.017954 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:18.017960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:18.018637 (+ 677us) server_negotiation.cc:407] Connection header received
0928 14:55:18.018833 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:18.018837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:18.018899 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:18.018970 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:55:18.019505 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.020181 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.021865 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.022653 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.022800 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:18.022877 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:18.023107 (+ 230us) server_negotiation.cc:299] Negotiation successful
0928 14:55:18.023247 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 14:55:18.023530 3223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:18.017727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.193:39321 (local address 127.14.241.195:55781)
0928 14:55:18.018526 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:18.018543 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:18.018684 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:18.019007 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:18.019011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:18.019034 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:55:18.019348 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:18.019357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.020357 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.020361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:18.021681 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:18.021692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.022811 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.022818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:18.022970 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:18.022993 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 14:55:18.023069 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:55:18.023184 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":101,"thread_start_us":386,"threads_started":1}
I20260928 14:55:18.024176 3021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a488a6d17d6c4f4c9260efefa827a042" candidate_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f786664c82f405387bf886abbd0cb8f" is_pre_election: true
I20260928 14:55:18.024478 2889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a488a6d17d6c4f4c9260efefa827a042" candidate_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529958ee62a84527923ebf0f4c0c926d" is_pre_election: true
I20260928 14:55:18.024701 3021 raft_consensus.cc:2454] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e75c29fb04d42a69fcbcb0c45520ada in term 0.
I20260928 14:55:18.024953 2889 raft_consensus.cc:2454] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e75c29fb04d42a69fcbcb0c45520ada in term 0.
I20260928 14:55:18.025411 3086 leader_election.cc:302] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e75c29fb04d42a69fcbcb0c45520ada, 8f786664c82f405387bf886abbd0cb8f; no voters:
I20260928 14:55:18.025781 3222 raft_consensus.cc:2781] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:55:18.025907 3222 raft_consensus.cc:482] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:55:18.026001 3222 raft_consensus.cc:3037] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:18.028117 3222 raft_consensus.cc:504] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38415 } } peers { permanent_uuid: "8f786664c82f405387bf886abbd0cb8f" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46035 } } peers { permanent_uuid: "529958ee62a84527923ebf0f4c0c926d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39321 } }
I20260928 14:55:18.028748 3222 leader_election.cc:288] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [CANDIDATE]: Term 1 election: Requested vote from peers 8f786664c82f405387bf886abbd0cb8f (127.14.241.194:46035), 529958ee62a84527923ebf0f4c0c926d (127.14.241.193:39321)
I20260928 14:55:18.029182 3021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a488a6d17d6c4f4c9260efefa827a042" candidate_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f786664c82f405387bf886abbd0cb8f"
I20260928 14:55:18.029213 2889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a488a6d17d6c4f4c9260efefa827a042" candidate_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529958ee62a84527923ebf0f4c0c926d"
I20260928 14:55:18.029368 3021 raft_consensus.cc:3037] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:18.029405 2889 raft_consensus.cc:3037] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:18.031519 2889 raft_consensus.cc:2454] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e75c29fb04d42a69fcbcb0c45520ada in term 1.
I20260928 14:55:18.031535 3021 raft_consensus.cc:2454] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e75c29fb04d42a69fcbcb0c45520ada in term 1.
I20260928 14:55:18.031908 3084 leader_election.cc:302] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e75c29fb04d42a69fcbcb0c45520ada, 529958ee62a84527923ebf0f4c0c926d; no voters:
I20260928 14:55:18.032202 3222 raft_consensus.cc:2781] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:18.032655 3222 raft_consensus.cc:686] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 1 LEADER]: Becoming Leader. State: Replica: 0e75c29fb04d42a69fcbcb0c45520ada, State: Running, Role: LEADER
I20260928 14:55:18.033170 3222 consensus_queue.cc:237] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38415 } } peers { permanent_uuid: "8f786664c82f405387bf886abbd0cb8f" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46035 } } peers { permanent_uuid: "529958ee62a84527923ebf0f4c0c926d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39321 } }
I20260928 14:55:18.037474 2746 catalog_manager.cc:5495] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e75c29fb04d42a69fcbcb0c45520ada (127.14.241.195). New cstate: current_term: 1 leader_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e75c29fb04d42a69fcbcb0c45520ada" member_type: VOTER last_known_addr { host: "127.14.241.195" port: 38415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f786664c82f405387bf886abbd0cb8f" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "529958ee62a84527923ebf0f4c0c926d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39321 } health_report { overall_health: UNKNOWN } } }
W20260928 14:55:18.048190 2935 tablet.cc:2369] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:55:18.052115 3209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:18.047125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53632 (local address 127.14.241.195:38415)
0928 14:55:18.047296 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:18.047301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:18.047320 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:55:18.047489 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:18.047493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:18.047568 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:18.047694 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:55:18.048119 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.048894 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.050236 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.050636 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.050723 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:18.051155 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:55:18.051643 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:55:18.051758 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:18.051794 (+ 36us) server_negotiation.cc:299] Negotiation successful
0928 14:55:18.051870 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 14:55:18.060177 3021 raft_consensus.cc:1261] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f [term 1 FOLLOWER]: Refusing update from remote peer 0e75c29fb04d42a69fcbcb0c45520ada: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:18.060177 2889 raft_consensus.cc:1261] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d [term 1 FOLLOWER]: Refusing update from remote peer 0e75c29fb04d42a69fcbcb0c45520ada: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:18.061170 3224 consensus_queue.cc:1035] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f786664c82f405387bf886abbd0cb8f" member_type: VOTER last_known_addr { host: "127.14.241.194" port: 46035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:18.061553 3222 consensus_queue.cc:1035] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "529958ee62a84527923ebf0f4c0c926d" member_type: VOTER last_known_addr { host: "127.14.241.193" port: 39321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:18.072535 3227 mvcc.cc:205] Tried to move back new op lower bound from 7334327574765035520 to 7334327574667370496. Current Snapshot: MvccSnapshot[applied={T|T < 7334327574765035520}]
I20260928 14:55:18.093505 2747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:18.093767 2747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:55:18.097445 2747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c063f600a71f4271b6e2d00814df2588: Sending DeleteTablet for 3 replicas of tablet a488a6d17d6c4f4c9260efefa827a042
I20260928 14:55:18.098680 3131 tablet_service.cc:1514] Processing DeleteTablet for tablet a488a6d17d6c4f4c9260efefa827a042 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:18 UTC) from {username='slave'} at 127.0.0.1:53622
I20260928 14:55:18.098727 2869 tablet_service.cc:1514] Processing DeleteTablet for tablet a488a6d17d6c4f4c9260efefa827a042 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:18 UTC) from {username='slave'} at 127.0.0.1:33190
I20260928 14:55:18.099792 3001 tablet_service.cc:1514] Processing DeleteTablet for tablet a488a6d17d6c4f4c9260efefa827a042 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:55:18 UTC) from {username='slave'} at 127.0.0.1:54626
I20260928 14:55:18.100626 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 2807
I20260928 14:55:18.108688 3238 tablet_replica.cc:330] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: stopping tablet replica
I20260928 14:55:18.109256 3238 raft_consensus.cc:2229] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:55:18.112567 3238 raft_consensus.cc:2258] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:18.112929 3239 tablet_replica.cc:330] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: stopping tablet replica
I20260928 14:55:18.113441 3239 raft_consensus.cc:2229] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:18.114022 3239 raft_consensus.cc:2258] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:18.116480 3238 ts_tablet_manager.cc:1905] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:18.116503 3239 ts_tablet_manager.cc:1905] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:18.117102 3237 tablet_replica.cc:330] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: stopping tablet replica
I20260928 14:55:18.117543 3237 raft_consensus.cc:2229] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:55:18.118070 3237 raft_consensus.cc:2258] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:55:18.121308 3237 ts_tablet_manager.cc:1905] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:55:18.122958 3239 ts_tablet_manager.cc:1918] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:18.123126 3239 log.cc:1196] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-1/wal/wals/a488a6d17d6c4f4c9260efefa827a042
I20260928 14:55:18.123617 3239 ts_tablet_manager.cc:1939] T a488a6d17d6c4f4c9260efefa827a042 P 8f786664c82f405387bf886abbd0cb8f: Deleting consensus metadata
I20260928 14:55:18.125639 2733 catalog_manager.cc:4841] TS 8f786664c82f405387bf886abbd0cb8f (127.14.241.194:46035): tablet a488a6d17d6c4f4c9260efefa827a042 (table test-table [id=9103956ca62f4455a992c94bf807325b]) successfully deleted
I20260928 14:55:18.127444 3237 ts_tablet_manager.cc:1918] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:18.127622 3237 log.cc:1196] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-0/wal/wals/a488a6d17d6c4f4c9260efefa827a042
I20260928 14:55:18.128113 3237 ts_tablet_manager.cc:1939] T a488a6d17d6c4f4c9260efefa827a042 P 529958ee62a84527923ebf0f4c0c926d: Deleting consensus metadata
I20260928 14:55:18.129673 2731 catalog_manager.cc:4841] TS 529958ee62a84527923ebf0f4c0c926d (127.14.241.193:39321): tablet a488a6d17d6c4f4c9260efefa827a042 (table test-table [id=9103956ca62f4455a992c94bf807325b]) successfully deleted
I20260928 14:55:18.131785 3238 ts_tablet_manager.cc:1918] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:55:18.131978 3238 log.cc:1196] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: Deleting WAL directory at /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790607004819300-15303-0/minicluster-data/ts-2/wal/wals/a488a6d17d6c4f4c9260efefa827a042
I20260928 14:55:18.132552 3238 ts_tablet_manager.cc:1939] T a488a6d17d6c4f4c9260efefa827a042 P 0e75c29fb04d42a69fcbcb0c45520ada: Deleting consensus metadata
I20260928 14:55:18.134073 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:18.101862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33192 (local address 127.14.241.193:39321)
0928 14:55:18.102044 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:18.102051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:18.114753 (+ 12702us) server_negotiation.cc:407] Connection header received
0928 14:55:18.116997 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:18.117004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:18.117122 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:18.117203 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:18.118127 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.119310 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.123320 (+ 4010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.124026 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.124128 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:18.132632 (+ 8504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:18.132646 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:18.132665 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:18.133271 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:18.133323 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:18.133346 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:18.133468 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:18.133640 (+ 172us) server_negotiation.cc:299] Negotiation successful
0928 14:55:18.133761 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 14:55:18.134455 2733 catalog_manager.cc:4841] TS 0e75c29fb04d42a69fcbcb0c45520ada (127.14.241.195:38415): tablet a488a6d17d6c4f4c9260efefa827a042 (table test-table [id=9103956ca62f4455a992c94bf807325b]) successfully deleted
I20260928 14:55:18.134761 2929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:18.225447 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 2807
I20260928 14:55:18.241350 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 2938
I20260928 14:55:18.254082 3213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:18.242151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.14.241.194:46035)
0928 14:55:18.242355 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:18.242361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:18.242382 (+ 21us) server_negotiation.cc:407] Connection header received
0928 14:55:18.242479 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:18.242484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:18.242593 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:18.242681 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:18.243108 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.243963 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.245382 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.245799 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.245920 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:18.252230 (+ 6310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:18.252241 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:18.252258 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:18.253127 (+ 869us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:18.253194 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:18.253224 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:18.253320 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:18.253608 (+ 288us) server_negotiation.cc:299] Negotiation successful
0928 14:55:18.253740 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260928 14:55:18.255219 3061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:18.345502 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 2938
I20260928 14:55:18.360823 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3069
I20260928 14:55:18.368521 3209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:18.361597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53642 (local address 127.14.241.195:38415)
0928 14:55:18.361757 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:18.361761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:18.361800 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:55:18.362040 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:18.362045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:18.362126 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:18.362201 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:18.362678 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.363535 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.365309 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:18.365721 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:18.365818 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:18.367070 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:18.367080 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:18.367095 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:18.367669 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:18.367749 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:18.367777 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:18.367867 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:18.368034 (+ 167us) server_negotiation.cc:299] Negotiation successful
0928 14:55:18.368124 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:55:18.369256 3191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:18.489787 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 3069
I20260928 14:55:18.505820 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 2716
I20260928 14:55:18.506929 2778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:18.616348 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 2716
2026-09-28T14:55:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2875 ms)
[----------] 6 tests from EncryptionPolicyTest (20805 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:18 dist-test-slave-vk40 krb5kdc[3247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:18 dist-test-slave-vk40 krb5kdc[3247](info): set up 2 sockets
Sep 28 14:55:18 dist-test-slave-vk40 krb5kdc[3247](info): commencing operation
krb5kdc: starting...
W20260928 14:55:19.639343 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:19 dist-test-slave-vk40 krb5kdc[3247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:19.890211 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45353
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:20.217748 3260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:20.218065 3260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:20.218201 3260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:20.225539 3260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:20.225657 3260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:20.225732 3260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:20.225821 3260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:20.225873 3260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:20.235906 3260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45353
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:20.237238 3260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:20.238544 3260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:20.249886 3265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:20.249946 3266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:20.250054 3268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:20.250825 3260 server_base.cc:1034] running on GCE node
I20260928 14:55:20.251551 3260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:20.253053 3260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:20.254266 3260 hybrid_clock.cc:648] HybridClock initialized: now 1790607320254218 us; error 44 us; skew 500 ppm
I20260928 14:55:20.254580 3260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:20 dist-test-slave-vk40 krb5kdc[3247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607320, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:20.257616 3260 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:20.259012 3260 webserver.cc:460] Webserver started at http://127.0.0.1:36135/ using document root <none> and password file <none>
I20260928 14:55:20.259636 3260 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:20.259748 3260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:20.260007 3260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:20.262946 3260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "eca669ceef3144a69b153fbfe6d89b1a"
format_stamp: "Formatted at 2026-09-28 14:55:20 on dist-test-slave-vk40"
server_key: "21bdc885f7dd7189eff589a71cb87e74"
server_key_iv: "6a7b764819a1c05a04de347cd9cba9fb"
server_key_version: "encryptionkey@0"
I20260928 14:55:20.263659 3260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "eca669ceef3144a69b153fbfe6d89b1a"
format_stamp: "Formatted at 2026-09-28 14:55:20 on dist-test-slave-vk40"
server_key: "21bdc885f7dd7189eff589a71cb87e74"
server_key_iv: "6a7b764819a1c05a04de347cd9cba9fb"
server_key_version: "encryptionkey@0"
I20260928 14:55:20.268857 3260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260928 14:55:20.272382 3275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:20.273746 3260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:55:20.273909 3260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "eca669ceef3144a69b153fbfe6d89b1a"
format_stamp: "Formatted at 2026-09-28 14:55:20 on dist-test-slave-vk40"
server_key: "21bdc885f7dd7189eff589a71cb87e74"
server_key_iv: "6a7b764819a1c05a04de347cd9cba9fb"
server_key_version: "encryptionkey@0"
I20260928 14:55:20.274080 3260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:20.295199 3260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:20.299522 3260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:20.299808 3260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:20.317647 3260 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45353
I20260928 14:55:20.317670 3326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45353 every 8 connection(s)
I20260928 14:55:20.319043 3260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:20.322849 3327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:20.323588 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 3260
I20260928 14:55:20.323783 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:20.324434 15303 external_mini_cluster.cc:1428] Setting key 0b97e2afddf75ba3c5dfa38d3692545e
I20260928 14:55:20.336692 3327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a: Bootstrap starting.
Sep 28 14:55:20 dist-test-slave-vk40 krb5kdc[3247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:55:20.340582 3327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:20.341670 3327 log.cc:824] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:20.344818 3327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a: No bootstrap required, opened a new log
I20260928 14:55:20.347815 3330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:20.326403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37285 (local address 127.0.0.1:45353)
0928 14:55:20.327178 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:20.327190 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:20.327234 (+ 44us) server_negotiation.cc:407] Connection header received
0928 14:55:20.328897 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:20.328921 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:20.329341 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:20.329662 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:20.331339 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:20.332668 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:20.334486 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:20.335500 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:20.335632 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:20.340320 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:20.340331 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:20.340346 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:20.340384 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:20.343611 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:20.344596 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:20.344602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:20.344608 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:20.344763 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:20.345230 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:20.345234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:20.345236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:20.345754 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:20.346003 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:20.346458 (+ 455us) server_negotiation.cc:299] Negotiation successful
0928 14:55:20.346758 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":358,"threads_started":1}
I20260928 14:55:20.351660 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:20.353402 3327 raft_consensus.cc:348] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca669ceef3144a69b153fbfe6d89b1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260928 14:55:20.353802 3327 raft_consensus.cc:374] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:20.353899 3327 raft_consensus.cc:729] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca669ceef3144a69b153fbfe6d89b1a, State: Initialized, Role: FOLLOWER
I20260928 14:55:20.354540 3327 consensus_queue.cc:260] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca669ceef3144a69b153fbfe6d89b1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260928 14:55:20.354817 3327 raft_consensus.cc:388] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:20.354950 3327 raft_consensus.cc:482] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:20.355134 3327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:20.358678 3327 raft_consensus.cc:504] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca669ceef3144a69b153fbfe6d89b1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260928 14:55:20.359369 3327 leader_election.cc:302] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eca669ceef3144a69b153fbfe6d89b1a; no voters:
I20260928 14:55:20.359999 3327 leader_election.cc:288] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:20.360077 3336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:20.362246 3327 sys_catalog.cc:564] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:20.362197 3336 raft_consensus.cc:686] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [term 1 LEADER]: Becoming Leader. State: Replica: eca669ceef3144a69b153fbfe6d89b1a, State: Running, Role: LEADER
I20260928 14:55:20.362857 3336 consensus_queue.cc:237] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca669ceef3144a69b153fbfe6d89b1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260928 14:55:20.367274 3338 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eca669ceef3144a69b153fbfe6d89b1a. Latest consensus state: current_term: 1 leader_uuid: "eca669ceef3144a69b153fbfe6d89b1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca669ceef3144a69b153fbfe6d89b1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } } }
I20260928 14:55:20.367169 3337 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eca669ceef3144a69b153fbfe6d89b1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca669ceef3144a69b153fbfe6d89b1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } } }
I20260928 14:55:20.367714 3338 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:20.367770 3337 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:20.369120 3345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:20.379904 3330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:20.361529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48467 (local address 127.0.0.1:45353)
0928 14:55:20.361697 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:20.361703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:20.361725 (+ 22us) server_negotiation.cc:407] Connection header received
0928 14:55:20.362042 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:20.362049 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:20.362134 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:20.362214 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:20.364301 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:20.365616 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:20.366911 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:20.367752 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:20.368639 (+ 887us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:20.371263 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:20.371271 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:20.371277 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:20.371321 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:20.375238 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:20.376383 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:20.376392 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:20.376397 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:20.376569 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:20.377170 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:20.377175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:20.377178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:20.377684 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:20.377954 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:20.379278 (+ 1324us) server_negotiation.cc:299] Negotiation successful
0928 14:55:20.379491 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 14:55:20.383324 3345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:20.393781 3345 catalog_manager.cc:1347] Generated new cluster ID: 9333334fbb9f4fd09da1fa3f46a53429
I20260928 14:55:20.393898 3345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:20.414032 3345 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:20.416113 3345 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:20.428782 3345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eca669ceef3144a69b153fbfe6d89b1a: Generated new TSK 0
I20260928 14:55:20.429672 3345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:20.441890 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3260
I20260928 14:55:20.443111 3322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:20.535625 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 3260
W20260928 14:55:20.550015 3245 connection.cc:535] client connection to 127.0.0.1:45353 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45353): Connection reset by peer (error 104)
2026-09-28T14:55:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1915 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:20 dist-test-slave-vk40 krb5kdc[3359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:20 dist-test-slave-vk40 krb5kdc[3359](info): set up 2 sockets
Sep 28 14:55:20 dist-test-slave-vk40 krb5kdc[3359](info): commencing operation
krb5kdc: starting...
W20260928 14:55:21.551323 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:21 dist-test-slave-vk40 krb5kdc[3359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:21.797968 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33375
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:22.138211 3372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:22.138501 3372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:22.138629 3372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:22.145974 3372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:22.146095 3372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:22.146184 3372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:22.146265 3372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:22.146333 3372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:22.156376 3372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33375
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:22.157676 3372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:22.158938 3372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:22.170431 3377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:22.170467 3380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:22.170500 3378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:22.171100 3372 server_base.cc:1034] running on GCE node
I20260928 14:55:22.171900 3372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:22.173341 3372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:22.174546 3372 hybrid_clock.cc:648] HybridClock initialized: now 1790607322174494 us; error 39 us; skew 500 ppm
I20260928 14:55:22.174885 3372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:22 dist-test-slave-vk40 krb5kdc[3359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:22.177958 3372 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:22.179346 3372 webserver.cc:460] Webserver started at http://127.0.0.1:43079/ using document root <none> and password file <none>
I20260928 14:55:22.180076 3372 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:22.180198 3372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:22.180524 3372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:22.183681 3372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "42050e50eba747bdb32bd4c8dec26cc7"
format_stamp: "Formatted at 2026-09-28 14:55:22 on dist-test-slave-vk40"
server_key: "b4d4f98120a3608368532bd12ee4c7a7"
server_key_iv: "9919ea982ad54f17198e8cee82032513"
server_key_version: "encryptionkey@0"
I20260928 14:55:22.184535 3372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "42050e50eba747bdb32bd4c8dec26cc7"
format_stamp: "Formatted at 2026-09-28 14:55:22 on dist-test-slave-vk40"
server_key: "b4d4f98120a3608368532bd12ee4c7a7"
server_key_iv: "9919ea982ad54f17198e8cee82032513"
server_key_version: "encryptionkey@0"
I20260928 14:55:22.189915 3372 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:55:22.193518 3387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:22.194948 3372 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 14:55:22.195115 3372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "42050e50eba747bdb32bd4c8dec26cc7"
format_stamp: "Formatted at 2026-09-28 14:55:22 on dist-test-slave-vk40"
server_key: "b4d4f98120a3608368532bd12ee4c7a7"
server_key_iv: "9919ea982ad54f17198e8cee82032513"
server_key_version: "encryptionkey@0"
I20260928 14:55:22.195305 3372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:22.230149 3372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:22.234571 3372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:22.234845 3372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:22.253650 3372 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33375
I20260928 14:55:22.253657 3439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33375 every 8 connection(s)
I20260928 14:55:22.255050 3372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:22.258839 3440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:22.261958 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 3372
I20260928 14:55:22.262136 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:22.262567 15303 external_mini_cluster.cc:1428] Setting key 9efed3ab0a894aa9427901fb04ceed8d
I20260928 14:55:22.272508 3440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7: Bootstrap starting.
I20260928 14:55:22.276486 3440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:22.277884 3440 log.cc:824] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:22.278369 3443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:22.264626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34219 (local address 127.0.0.1:33375)
0928 14:55:22.265439 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:22.265456 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:22.265508 (+ 52us) server_negotiation.cc:407] Connection header received
0928 14:55:22.267818 (+ 2310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:22.267848 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:22.268477 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:22.268950 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:22.269533 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:22.271506 (+ 1973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:22.273016 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:22.274029 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:22.274253 (+ 224us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:22.275435 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:22.275446 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:22.275466 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:22.276201 (+ 735us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:22.276317 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:22.276352 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:22.276529 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:22.276799 (+ 270us) server_negotiation.cc:299] Negotiation successful
0928 14:55:22.277017 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":318,"threads_started":1}
I20260928 14:55:22.281106 3440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7: No bootstrap required, opened a new log
I20260928 14:55:22.282584 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:22.291184 3440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42050e50eba747bdb32bd4c8dec26cc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } }
I20260928 14:55:22.291577 3440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:22.291682 3440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42050e50eba747bdb32bd4c8dec26cc7, State: Initialized, Role: FOLLOWER
I20260928 14:55:22.292333 3440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42050e50eba747bdb32bd4c8dec26cc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } }
I20260928 14:55:22.292600 3440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:22.292768 3440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:22.292943 3440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:22.294553 3443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:22.287895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39425 (local address 127.0.0.1:33375)
0928 14:55:22.288074 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:22.288080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:22.288502 (+ 422us) server_negotiation.cc:407] Connection header received
0928 14:55:22.288664 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:22.288669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:22.288749 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:22.288873 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:22.289360 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:22.290524 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:22.292006 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:22.292502 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:22.292625 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:22.293828 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:22.293833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:22.293836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:22.293857 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:22.293915 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:22.293931 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:22.294039 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:22.294230 (+ 191us) server_negotiation.cc:299] Negotiation successful
0928 14:55:22.294288 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:55:22.295460 3440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42050e50eba747bdb32bd4c8dec26cc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } }
I20260928 14:55:22.296023 3440 leader_election.cc:302] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42050e50eba747bdb32bd4c8dec26cc7; no voters:
I20260928 14:55:22.296650 3440 leader_election.cc:288] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:22.296789 3450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:22.297459 3450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [term 1 LEADER]: Becoming Leader. State: Replica: 42050e50eba747bdb32bd4c8dec26cc7, State: Running, Role: LEADER
I20260928 14:55:22.297972 3450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42050e50eba747bdb32bd4c8dec26cc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } }
I20260928 14:55:22.298696 3440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:22.300741 3451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42050e50eba747bdb32bd4c8dec26cc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42050e50eba747bdb32bd4c8dec26cc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } } }
I20260928 14:55:22.301112 3451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:22.305545 3452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42050e50eba747bdb32bd4c8dec26cc7. Latest consensus state: current_term: 1 leader_uuid: "42050e50eba747bdb32bd4c8dec26cc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42050e50eba747bdb32bd4c8dec26cc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } } }
I20260928 14:55:22.305771 3456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:22.305816 3452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:22.310930 3456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:22.320389 3456 catalog_manager.cc:1347] Generated new cluster ID: ed4fea908c014f4d835c4ab3886fc3bd
I20260928 14:55:22.320514 3456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:22.339926 3456 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:22.341240 3456 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:22.362089 3456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42050e50eba747bdb32bd4c8dec26cc7: Generated new TSK 0
I20260928 14:55:22.362986 3456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:22.373800 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3372
W20260928 14:55:22.375361 3435 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34219
W20260928 14:55:22.376235 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3372: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:22.376447 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 3372
W20260928 14:55:22.390465 3357 connection.cc:535] client connection to 127.0.0.1:33375 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33375): Connection reset by peer (error 104)
2026-09-28T14:55:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1840 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:22 dist-test-slave-vk40 krb5kdc[3471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:22 dist-test-slave-vk40 krb5kdc[3471](info): set up 2 sockets
Sep 28 14:55:22 dist-test-slave-vk40 krb5kdc[3471](info): commencing operation
krb5kdc: starting...
W20260928 14:55:23.411345 15303 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.004s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:23 dist-test-slave-vk40 krb5kdc[3471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:23.663980 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42425
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:23.988054 3485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:23.988405 3485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:23.988539 3485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:23.995996 3485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:23.996138 3485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:23.996227 3485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:23.996330 3485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:23.996400 3485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:24.006488 3485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42425
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:24.007872 3485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:24.009758 3485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:24.021473 3491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:24.021519 3490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:24.021579 3493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:24.022218 3485 server_base.cc:1034] running on GCE node
I20260928 14:55:24.022984 3485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:24.024479 3485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:24.025697 3485 hybrid_clock.cc:648] HybridClock initialized: now 1790607324025647 us; error 40 us; skew 500 ppm
I20260928 14:55:24.026049 3485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:24 dist-test-slave-vk40 krb5kdc[3471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607324, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:24.032997 3485 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:24.034471 3485 webserver.cc:460] Webserver started at http://127.0.0.1:41697/ using document root <none> and password file <none>
I20260928 14:55:24.035125 3485 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:24.035229 3485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:24.035483 3485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:24.038503 3485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "a168863d62bb42cba413bd470b6a7b02"
format_stamp: "Formatted at 2026-09-28 14:55:24 on dist-test-slave-vk40"
server_key: "fa507a1cb25749cc6b33cb12e6db2dc8"
server_key_iv: "63ba1e18d36d4e892bea0cef7f530401"
server_key_version: "encryptionkey@0"
I20260928 14:55:24.039217 3485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "a168863d62bb42cba413bd470b6a7b02"
format_stamp: "Formatted at 2026-09-28 14:55:24 on dist-test-slave-vk40"
server_key: "fa507a1cb25749cc6b33cb12e6db2dc8"
server_key_iv: "63ba1e18d36d4e892bea0cef7f530401"
server_key_version: "encryptionkey@0"
I20260928 14:55:24.044262 3485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260928 14:55:24.048026 3500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:24.049448 3485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260928 14:55:24.049611 3485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "a168863d62bb42cba413bd470b6a7b02"
format_stamp: "Formatted at 2026-09-28 14:55:24 on dist-test-slave-vk40"
server_key: "fa507a1cb25749cc6b33cb12e6db2dc8"
server_key_iv: "63ba1e18d36d4e892bea0cef7f530401"
server_key_version: "encryptionkey@0"
I20260928 14:55:24.049798 3485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:24.055621 3485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:24.059854 3485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:24.060127 3485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:24.077299 3485 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42425
I20260928 14:55:24.077348 3551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42425 every 8 connection(s)
I20260928 14:55:24.078704 3485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:24.082376 3552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:24.087936 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 3485
I20260928 14:55:24.088089 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:24.088542 15303 external_mini_cluster.cc:1428] Setting key d07a5036987d63e64119e138ccf107e2
I20260928 14:55:24.093750 3552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02: Bootstrap starting.
I20260928 14:55:24.096589 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:24.090231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42821 (local address 127.0.0.1:42425)
0928 14:55:24.091402 (+ 1171us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:24.091420 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:24.091481 (+ 61us) server_negotiation.cc:407] Connection header received
0928 14:55:24.093275 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:24.093299 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:24.093658 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:24.094022 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:24.095000 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:24.095006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:24.095019 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:24.095033 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:24.095086 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:24.095106 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:24.095266 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:24.095397 (+ 131us) server_negotiation.cc:299] Negotiation successful
0928 14:55:24.095585 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":461,"threads_started":1}
I20260928 14:55:24.097316 3552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:24.098385 3552 log.cc:824] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:24.100220 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:24.101353 3552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02: No bootstrap required, opened a new log
I20260928 14:55:24.106726 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:24.104367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46071 (local address 127.0.0.1:42425)
0928 14:55:24.104560 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:24.104567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:24.104918 (+ 351us) server_negotiation.cc:407] Connection header received
0928 14:55:24.105078 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:24.105081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:24.105127 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:24.105202 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:24.106081 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:24.106085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:24.106088 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:24.106103 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:24.106146 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:24.106161 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:24.106252 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:24.106373 (+ 121us) server_negotiation.cc:299] Negotiation successful
0928 14:55:24.106428 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:55:24.109817 3552 raft_consensus.cc:348] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a168863d62bb42cba413bd470b6a7b02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42425 } }
I20260928 14:55:24.110200 3552 raft_consensus.cc:374] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:24.110299 3552 raft_consensus.cc:729] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a168863d62bb42cba413bd470b6a7b02, State: Initialized, Role: FOLLOWER
I20260928 14:55:24.110922 3552 consensus_queue.cc:260] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a168863d62bb42cba413bd470b6a7b02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42425 } }
I20260928 14:55:24.111215 3552 raft_consensus.cc:388] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:24.111351 3552 raft_consensus.cc:482] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:24.111521 3552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:24.115159 3552 raft_consensus.cc:504] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a168863d62bb42cba413bd470b6a7b02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42425 } }
I20260928 14:55:24.115818 3552 leader_election.cc:302] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a168863d62bb42cba413bd470b6a7b02; no voters:
I20260928 14:55:24.116679 3552 leader_election.cc:288] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:24.116807 3562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:24.117471 3562 raft_consensus.cc:686] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [term 1 LEADER]: Becoming Leader. State: Replica: a168863d62bb42cba413bd470b6a7b02, State: Running, Role: LEADER
I20260928 14:55:24.118201 3562 consensus_queue.cc:237] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a168863d62bb42cba413bd470b6a7b02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42425 } }
I20260928 14:55:24.119908 3552 sys_catalog.cc:564] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:24.121426 3564 sys_catalog.cc:455] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a168863d62bb42cba413bd470b6a7b02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a168863d62bb42cba413bd470b6a7b02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42425 } } }
I20260928 14:55:24.121985 3564 sys_catalog.cc:458] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:24.121374 3563 sys_catalog.cc:455] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a168863d62bb42cba413bd470b6a7b02. Latest consensus state: current_term: 1 leader_uuid: "a168863d62bb42cba413bd470b6a7b02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a168863d62bb42cba413bd470b6a7b02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42425 } } }
I20260928 14:55:24.122378 3563 sys_catalog.cc:458] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:24.124586 3567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:24.130362 3567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:24.145569 3567 catalog_manager.cc:1347] Generated new cluster ID: 4c167a6287ba473692e390c8f522df94
I20260928 14:55:24.145709 3567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:24.164083 3567 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:24.164542 3567 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:24.170207 3567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a168863d62bb42cba413bd470b6a7b02: Generated new TSK 0
I20260928 14:55:24.170989 3567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:24.178295 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3485
W20260928 14:55:24.179668 3547 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42821
W20260928 14:55:24.180476 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3485: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:24.180626 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 3485
2026-09-28T14:55:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1803 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:24 dist-test-slave-vk40 krb5kdc[3583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:24 dist-test-slave-vk40 krb5kdc[3583](info): set up 2 sockets
Sep 28 14:55:24 dist-test-slave-vk40 krb5kdc[3583](info): commencing operation
krb5kdc: starting...
W20260928 14:55:25.195335 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:25 dist-test-slave-vk40 krb5kdc[3583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:25.452059 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38363
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:25.776808 3596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:25.777112 3596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:25.777257 3596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:25.784687 3596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:25.784845 3596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:25.784921 3596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:25.785002 3596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:25.785059 3596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:25.795076 3596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:25.796391 3596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:25.797660 3596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:25.809286 3601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:25.809314 3604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:25.809240 3602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:25.810177 3596 server_base.cc:1034] running on GCE node
I20260928 14:55:25.810920 3596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:25.812371 3596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:25.813568 3596 hybrid_clock.cc:648] HybridClock initialized: now 1790607325813506 us; error 52 us; skew 500 ppm
I20260928 14:55:25.813915 3596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:25 dist-test-slave-vk40 krb5kdc[3583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607325, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:25.816959 3596 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:25.818447 3596 webserver.cc:460] Webserver started at http://127.0.0.1:43477/ using document root <none> and password file <none>
I20260928 14:55:25.819115 3596 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:25.819226 3596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:25.819509 3596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:25.822559 3596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "75a50affb4bf4e32bceee39e4c2a828d"
format_stamp: "Formatted at 2026-09-28 14:55:25 on dist-test-slave-vk40"
server_key: "7a735bc28e099b424ccdcdbec8890eda"
server_key_iv: "187d034631404c7a8dc6b270fc9f8c0f"
server_key_version: "encryptionkey@0"
I20260928 14:55:25.823316 3596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "75a50affb4bf4e32bceee39e4c2a828d"
format_stamp: "Formatted at 2026-09-28 14:55:25 on dist-test-slave-vk40"
server_key: "7a735bc28e099b424ccdcdbec8890eda"
server_key_iv: "187d034631404c7a8dc6b270fc9f8c0f"
server_key_version: "encryptionkey@0"
I20260928 14:55:25.828449 3596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260928 14:55:25.831825 3611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:25.833166 3596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:55:25.833333 3596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "75a50affb4bf4e32bceee39e4c2a828d"
format_stamp: "Formatted at 2026-09-28 14:55:25 on dist-test-slave-vk40"
server_key: "7a735bc28e099b424ccdcdbec8890eda"
server_key_iv: "187d034631404c7a8dc6b270fc9f8c0f"
server_key_version: "encryptionkey@0"
I20260928 14:55:25.833501 3596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:25.851033 3596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:25.854993 3596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:25.855253 3596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:25.872926 3596 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38363
I20260928 14:55:25.872963 3663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38363 every 8 connection(s)
I20260928 14:55:25.874231 3596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:25.876015 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 3596
I20260928 14:55:25.876189 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:25.876782 15303 external_mini_cluster.cc:1428] Setting key 505971e8a423b16866e7e794e2a324f0
I20260928 14:55:25.878654 3664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 28 14:55:25 dist-test-slave-vk40 krb5kdc[3583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:55:25.891505 3664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d: Bootstrap starting.
I20260928 14:55:25.894761 3664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:25.895771 3664 log.cc:824] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:25.898562 3664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d: No bootstrap required, opened a new log
I20260928 14:55:25.899022 3667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:25.878512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33399 (local address 127.0.0.1:38363)
0928 14:55:25.879239 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:25.879250 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:25.879290 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:55:25.880982 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:25.881006 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:25.881426 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:25.881762 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:25.883561 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:25.884992 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:25.886767 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:25.887451 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:25.887578 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:25.892064 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:25.892072 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:25.892087 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:25.892124 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:25.895096 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:25.895737 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:25.895743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:25.895749 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:25.895922 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:25.896330 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:25.896335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:25.896338 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:25.896880 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:25.897167 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:25.897620 (+ 453us) server_negotiation.cc:299] Negotiation successful
0928 14:55:25.898029 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":327,"threads_started":1}
I20260928 14:55:25.902534 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:25.906337 3664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a50affb4bf4e32bceee39e4c2a828d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } }
I20260928 14:55:25.906780 3664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:25.906890 3664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a50affb4bf4e32bceee39e4c2a828d, State: Initialized, Role: FOLLOWER
I20260928 14:55:25.907545 3664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a50affb4bf4e32bceee39e4c2a828d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } }
I20260928 14:55:25.907836 3664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:25.907972 3664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:25.908146 3664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:25.910894 3664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a50affb4bf4e32bceee39e4c2a828d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } }
I20260928 14:55:25.911485 3664 leader_election.cc:302] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75a50affb4bf4e32bceee39e4c2a828d; no voters:
I20260928 14:55:25.912175 3664 leader_election.cc:288] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:25.912343 3674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:25.913326 3674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [term 1 LEADER]: Becoming Leader. State: Replica: 75a50affb4bf4e32bceee39e4c2a828d, State: Running, Role: LEADER
I20260928 14:55:25.914197 3674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a50affb4bf4e32bceee39e4c2a828d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } }
I20260928 14:55:25.914676 3664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:25.917094 3675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75a50affb4bf4e32bceee39e4c2a828d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a50affb4bf4e32bceee39e4c2a828d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } } }
I20260928 14:55:25.917472 3675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:25.922400 3676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75a50affb4bf4e32bceee39e4c2a828d. Latest consensus state: current_term: 1 leader_uuid: "75a50affb4bf4e32bceee39e4c2a828d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a50affb4bf4e32bceee39e4c2a828d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38363 } } }
I20260928 14:55:25.923262 3676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:25.923791 3667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:25.906677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56111 (local address 127.0.0.1:38363)
0928 14:55:25.906882 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:25.906889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:25.907323 (+ 434us) server_negotiation.cc:407] Connection header received
0928 14:55:25.907479 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:25.907483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:25.907545 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:25.907622 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:25.909574 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:25.910352 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:25.912014 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:25.912765 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:25.912915 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:25.915338 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:25.915343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:25.915347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:25.915380 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:25.919568 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:25.920574 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:25.920580 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:25.920584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:25.920827 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:25.921292 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:25.921297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:25.921301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:25.921704 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:25.922990 (+ 1286us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:25.923159 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 14:55:25.923417 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260928 14:55:25.925714 3680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:25.931334 3680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:25.940426 3680 catalog_manager.cc:1347] Generated new cluster ID: 32c656d4c33e49a2a2f6cfc187e0b5b5
I20260928 14:55:25.940538 3680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:25.973253 3680 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:25.974539 3680 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:25.983160 3680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75a50affb4bf4e32bceee39e4c2a828d: Generated new TSK 0
I20260928 14:55:25.983975 3680 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:25.992967 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3596
I20260928 14:55:25.994189 3659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:26.087975 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 3596
2026-09-28T14:55:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1908 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:26 dist-test-slave-vk40 krb5kdc[3696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:26 dist-test-slave-vk40 krb5kdc[3696](info): set up 2 sockets
Sep 28 14:55:26 dist-test-slave-vk40 krb5kdc[3696](info): commencing operation
krb5kdc: starting...
W20260928 14:55:27.099309 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:27 dist-test-slave-vk40 krb5kdc[3696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:27.347324 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36735
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:27.671600 3709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:27.671934 3709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:27.672076 3709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:27.679440 3709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:27.679567 3709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:27.679641 3709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:27.679729 3709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:27.679786 3709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:27.689874 3709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:27.691138 3709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:27.692478 3709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:27.703778 3717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:27.703740 3715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:27.703828 3714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:27.704355 3709 server_base.cc:1034] running on GCE node
I20260928 14:55:27.705137 3709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:27.706525 3709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:27.707741 3709 hybrid_clock.cc:648] HybridClock initialized: now 1790607327707702 us; error 55 us; skew 500 ppm
I20260928 14:55:27.708068 3709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:27 dist-test-slave-vk40 krb5kdc[3696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607327, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:27.711093 3709 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:27.712615 3709 webserver.cc:460] Webserver started at http://127.0.0.1:33737/ using document root <none> and password file <none>
I20260928 14:55:27.713248 3709 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:27.713344 3709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:27.713590 3709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:27.716607 3709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "029c56908e244fca880fa1b8a8352db1"
format_stamp: "Formatted at 2026-09-28 14:55:27 on dist-test-slave-vk40"
server_key: "123d2466bf9b0665a8187e8071281ef8"
server_key_iv: "ca08dc69331dda90ae145c73a3927a4b"
server_key_version: "encryptionkey@0"
I20260928 14:55:27.717337 3709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "029c56908e244fca880fa1b8a8352db1"
format_stamp: "Formatted at 2026-09-28 14:55:27 on dist-test-slave-vk40"
server_key: "123d2466bf9b0665a8187e8071281ef8"
server_key_iv: "ca08dc69331dda90ae145c73a3927a4b"
server_key_version: "encryptionkey@0"
I20260928 14:55:27.722313 3709 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260928 14:55:27.725742 3724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:27.727097 3709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260928 14:55:27.727229 3709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "029c56908e244fca880fa1b8a8352db1"
format_stamp: "Formatted at 2026-09-28 14:55:27 on dist-test-slave-vk40"
server_key: "123d2466bf9b0665a8187e8071281ef8"
server_key_iv: "ca08dc69331dda90ae145c73a3927a4b"
server_key_version: "encryptionkey@0"
I20260928 14:55:27.727389 3709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:27.744465 3709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:27.748481 3709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:27.748716 3709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:27.765486 3709 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36735
I20260928 14:55:27.765518 3776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36735 every 8 connection(s)
I20260928 14:55:27.766821 3709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:27.770489 3777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:27.770781 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 3709
I20260928 14:55:27.770965 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:27.771521 15303 external_mini_cluster.cc:1428] Setting key 38170e4c95b12c4f823254aa5b0234d2
I20260928 14:55:27.784508 3777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1: Bootstrap starting.
I20260928 14:55:27.784610 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:27.773357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52447 (local address 127.0.0.1:36735)
0928 14:55:27.774211 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:27.774223 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:27.774266 (+ 43us) server_negotiation.cc:407] Connection header received
0928 14:55:27.775811 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:27.775835 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:27.776259 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:27.776659 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:27.777204 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:27.778529 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:27.780253 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:27.780968 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:27.781106 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:27.782262 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:27.782269 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:27.782283 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:27.782845 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:27.782929 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:27.782954 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:27.783041 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:27.783243 (+ 202us) server_negotiation.cc:299] Negotiation successful
0928 14:55:27.783505 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":407,"threads_started":1}
I20260928 14:55:27.787917 3777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:27.788153 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:27.788966 3777 log.cc:824] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:27.791786 3777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1: No bootstrap required, opened a new log
I20260928 14:55:27.798606 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:27.792561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43715 (local address 127.0.0.1:36735)
0928 14:55:27.792728 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:27.792734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:27.793082 (+ 348us) server_negotiation.cc:407] Connection header received
0928 14:55:27.793228 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:27.793237 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:27.793317 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:27.793422 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:27.793846 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:27.794925 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:27.796127 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:27.796855 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:27.796977 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:27.797919 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:27.797925 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:27.797930 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:27.797949 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:27.797994 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:27.798018 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:27.798107 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:27.798188 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 14:55:27.798282 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:55:27.801007 3777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029c56908e244fca880fa1b8a8352db1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36735 } }
I20260928 14:55:27.801367 3777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:27.801440 3777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 029c56908e244fca880fa1b8a8352db1, State: Initialized, Role: FOLLOWER
I20260928 14:55:27.801942 3777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029c56908e244fca880fa1b8a8352db1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36735 } }
I20260928 14:55:27.802136 3777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:27.802237 3777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:27.802349 3777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:27.804445 3777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029c56908e244fca880fa1b8a8352db1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36735 } }
I20260928 14:55:27.804972 3777 leader_election.cc:302] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 029c56908e244fca880fa1b8a8352db1; no voters:
I20260928 14:55:27.805593 3777 leader_election.cc:288] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:27.805697 3787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:27.806288 3787 raft_consensus.cc:686] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [term 1 LEADER]: Becoming Leader. State: Replica: 029c56908e244fca880fa1b8a8352db1, State: Running, Role: LEADER
I20260928 14:55:27.806902 3787 consensus_queue.cc:237] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029c56908e244fca880fa1b8a8352db1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36735 } }
I20260928 14:55:27.807847 3777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:27.809674 3789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 029c56908e244fca880fa1b8a8352db1. Latest consensus state: current_term: 1 leader_uuid: "029c56908e244fca880fa1b8a8352db1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029c56908e244fca880fa1b8a8352db1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36735 } } }
I20260928 14:55:27.810043 3789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:27.810807 3788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "029c56908e244fca880fa1b8a8352db1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029c56908e244fca880fa1b8a8352db1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36735 } } }
I20260928 14:55:27.811211 3788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:27.812431 3794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:27.817623 3794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:27.825903 3794 catalog_manager.cc:1347] Generated new cluster ID: 4ce4249c60914fb28398d41e65e72722
I20260928 14:55:27.826013 3794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:27.846518 3794 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:27.848480 3794 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:27.864701 3794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 029c56908e244fca880fa1b8a8352db1: Generated new TSK 0
I20260928 14:55:27.865780 3794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:27.875321 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3709
W20260928 14:55:27.876535 3772 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52447
W20260928 14:55:27.877270 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3709: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:27.877408 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 3709
2026-09-28T14:55:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1788 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:27 dist-test-slave-vk40 krb5kdc[3808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:27 dist-test-slave-vk40 krb5kdc[3808](info): set up 2 sockets
Sep 28 14:55:27 dist-test-slave-vk40 krb5kdc[3808](info): commencing operation
krb5kdc: starting...
W20260928 14:55:28.891332 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.007s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:29 dist-test-slave-vk40 krb5kdc[3808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:29.143424 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:29.473057 3821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:29.473349 3821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:29.473461 3821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:29.480746 3821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:29.480872 3821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:29.480926 3821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:29.480988 3821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:29.481032 3821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:29.490978 3821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:29.492180 3821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:29.493525 3821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:29.504634 3826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:29.504573 3827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:29.504603 3829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:29.505682 3821 server_base.cc:1034] running on GCE node
I20260928 14:55:29.506379 3821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:29.507702 3821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:29.508895 3821 hybrid_clock.cc:648] HybridClock initialized: now 1790607329508839 us; error 48 us; skew 500 ppm
I20260928 14:55:29.509222 3821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:29 dist-test-slave-vk40 krb5kdc[3808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607329, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:29.517751 3821 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:29.519408 3821 webserver.cc:460] Webserver started at http://127.0.0.1:40393/ using document root <none> and password file <none>
I20260928 14:55:29.520114 3821 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:29.520221 3821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:29.520543 3821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:29.523669 3821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "2ed1d84722a04f31b4143afe1a9ae1cd"
format_stamp: "Formatted at 2026-09-28 14:55:29 on dist-test-slave-vk40"
server_key: "19679af27bb5b842254a6918aa5d1a90"
server_key_iv: "575143ac230578debb73ff32e9f5ae9c"
server_key_version: "encryptionkey@0"
I20260928 14:55:29.524518 3821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "2ed1d84722a04f31b4143afe1a9ae1cd"
format_stamp: "Formatted at 2026-09-28 14:55:29 on dist-test-slave-vk40"
server_key: "19679af27bb5b842254a6918aa5d1a90"
server_key_iv: "575143ac230578debb73ff32e9f5ae9c"
server_key_version: "encryptionkey@0"
I20260928 14:55:29.529963 3821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:55:29.533560 3836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:29.534924 3821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:55:29.535064 3821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "2ed1d84722a04f31b4143afe1a9ae1cd"
format_stamp: "Formatted at 2026-09-28 14:55:29 on dist-test-slave-vk40"
server_key: "19679af27bb5b842254a6918aa5d1a90"
server_key_iv: "575143ac230578debb73ff32e9f5ae9c"
server_key_version: "encryptionkey@0"
I20260928 14:55:29.535243 3821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:29.541016 3821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:29.545117 3821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:29.545368 3821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:29.562572 3821 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38299
I20260928 14:55:29.562610 3888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38299 every 8 connection(s)
I20260928 14:55:29.563918 3821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:29.567035 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 3821
I20260928 14:55:29.567184 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:29.567587 15303 external_mini_cluster.cc:1428] Setting key 334db0d8519f92680f604332807730ba
I20260928 14:55:29.567655 3889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:29.575616 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:29.569307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43517 (local address 127.0.0.1:38299)
0928 14:55:29.570108 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:29.570122 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:29.570177 (+ 55us) server_negotiation.cc:407] Connection header received
0928 14:55:29.572385 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:29.572409 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:29.572754 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:29.573089 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:29.574099 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:29.574105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:29.574118 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:29.574132 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:29.574204 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:29.574224 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:29.574389 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:29.574489 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 14:55:29.574670 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":309,"threads_started":1}
I20260928 14:55:29.579136 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:29.581277 3889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd: Bootstrap starting.
I20260928 14:55:29.584596 3889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:29.585602 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:29.583232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41209 (local address 127.0.0.1:38299)
0928 14:55:29.583414 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:29.583419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:29.583756 (+ 337us) server_negotiation.cc:407] Connection header received
0928 14:55:29.583920 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:29.583924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:29.583973 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:29.584048 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:29.585011 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:29.585019 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:29.585022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:29.585038 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:29.585079 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:29.585096 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:29.585183 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:29.585296 (+ 113us) server_negotiation.cc:299] Negotiation successful
0928 14:55:29.585341 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 14:55:29.585922 3889 log.cc:824] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:29.588807 3889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd: No bootstrap required, opened a new log
I20260928 14:55:29.595924 3889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed1d84722a04f31b4143afe1a9ae1cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260928 14:55:29.596319 3889 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:29.596417 3889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed1d84722a04f31b4143afe1a9ae1cd, State: Initialized, Role: FOLLOWER
I20260928 14:55:29.597024 3889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed1d84722a04f31b4143afe1a9ae1cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260928 14:55:29.597256 3889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:29.597383 3889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:29.597527 3889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:29.599627 3889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed1d84722a04f31b4143afe1a9ae1cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260928 14:55:29.600106 3889 leader_election.cc:302] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ed1d84722a04f31b4143afe1a9ae1cd; no voters:
I20260928 14:55:29.600673 3889 leader_election.cc:288] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:29.600802 3899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:29.601406 3899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [term 1 LEADER]: Becoming Leader. State: Replica: 2ed1d84722a04f31b4143afe1a9ae1cd, State: Running, Role: LEADER
I20260928 14:55:29.601935 3899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed1d84722a04f31b4143afe1a9ae1cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260928 14:55:29.602607 3889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:29.604449 3901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ed1d84722a04f31b4143afe1a9ae1cd. Latest consensus state: current_term: 1 leader_uuid: "2ed1d84722a04f31b4143afe1a9ae1cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed1d84722a04f31b4143afe1a9ae1cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } } }
I20260928 14:55:29.604971 3901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:29.605950 3900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ed1d84722a04f31b4143afe1a9ae1cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed1d84722a04f31b4143afe1a9ae1cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } } }
I20260928 14:55:29.608403 3900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:29.609511 3908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:29.619443 3908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:29.628652 3908 catalog_manager.cc:1347] Generated new cluster ID: aa1ffb97377d4517a5f493ac770e812e
I20260928 14:55:29.628785 3908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:29.639609 3908 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:29.640115 3908 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:29.653020 3908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ed1d84722a04f31b4143afe1a9ae1cd: Generated new TSK 0
I20260928 14:55:29.653820 3908 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:29.666903 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3821
W20260928 14:55:29.668090 3884 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43517
W20260928 14:55:29.668788 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3821: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:29.668915 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 3821
2026-09-28T14:55:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1791 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:29 dist-test-slave-vk40 krb5kdc[3920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:29 dist-test-slave-vk40 krb5kdc[3920](info): set up 2 sockets
Sep 28 14:55:29 dist-test-slave-vk40 krb5kdc[3920](info): commencing operation
krb5kdc: starting...
W20260928 14:55:30.691253 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:30 dist-test-slave-vk40 krb5kdc[3920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:30.940727 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39181
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:31.269650 3933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:31.269958 3933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:31.270092 3933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:31.278064 3933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:31.278195 3933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:31.278373 3933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:31.278476 3933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:31.278540 3933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:31.289341 3933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:31.290683 3933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:31.292043 3933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:31.303541 3939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:31.303571 3941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:31.304059 3938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:31.304402 3933 server_base.cc:1034] running on GCE node
I20260928 14:55:31.305125 3933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:31.306527 3933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:31.307720 3933 hybrid_clock.cc:648] HybridClock initialized: now 1790607331307674 us; error 37 us; skew 500 ppm
I20260928 14:55:31.308044 3933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:31 dist-test-slave-vk40 krb5kdc[3920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607331, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:31.311040 3933 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:31.312480 3933 webserver.cc:460] Webserver started at http://127.0.0.1:40351/ using document root <none> and password file <none>
I20260928 14:55:31.313125 3933 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:31.313242 3933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:31.313530 3933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:31.316545 3933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "da783e998d3441ba9625df7d853fc3ad"
format_stamp: "Formatted at 2026-09-28 14:55:31 on dist-test-slave-vk40"
server_key: "2378a4b7bea11927083b1f33068e8ad9"
server_key_iv: "b597503fed72eef04eac29b761eef8d9"
server_key_version: "encryptionkey@0"
I20260928 14:55:31.317302 3933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "da783e998d3441ba9625df7d853fc3ad"
format_stamp: "Formatted at 2026-09-28 14:55:31 on dist-test-slave-vk40"
server_key: "2378a4b7bea11927083b1f33068e8ad9"
server_key_iv: "b597503fed72eef04eac29b761eef8d9"
server_key_version: "encryptionkey@0"
I20260928 14:55:31.322454 3933 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:31.325997 3948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:31.327324 3933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:55:31.327497 3933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "da783e998d3441ba9625df7d853fc3ad"
format_stamp: "Formatted at 2026-09-28 14:55:31 on dist-test-slave-vk40"
server_key: "2378a4b7bea11927083b1f33068e8ad9"
server_key_iv: "b597503fed72eef04eac29b761eef8d9"
server_key_version: "encryptionkey@0"
I20260928 14:55:31.327683 3933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:31.348161 3933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:31.352454 3933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:31.352723 3933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:31.372992 3933 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39181
I20260928 14:55:31.373013 3999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39181 every 8 connection(s)
I20260928 14:55:31.374926 3933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:31.378957 4000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:31.384387 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 3933
I20260928 14:55:31.384567 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:31.385053 15303 external_mini_cluster.cc:1428] Setting key 09528e9d948b330d221135192ca4a0f3
I20260928 14:55:31.390067 4000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad: Bootstrap starting.
I20260928 14:55:31.393189 4000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:31.394124 4000 log.cc:824] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:31.396968 4000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad: No bootstrap required, opened a new log
Sep 28 14:55:31 dist-test-slave-vk40 krb5kdc[3920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:55:31.405819 4000 raft_consensus.cc:348] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da783e998d3441ba9625df7d853fc3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } }
I20260928 14:55:31.406243 4000 raft_consensus.cc:374] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:31.406352 4000 raft_consensus.cc:729] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da783e998d3441ba9625df7d853fc3ad, State: Initialized, Role: FOLLOWER
I20260928 14:55:31.406965 4000 consensus_queue.cc:260] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da783e998d3441ba9625df7d853fc3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } }
I20260928 14:55:31.407209 4000 raft_consensus.cc:388] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:31.407265 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:31.386933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:48539 (local address 127.0.0.1:39181)
0928 14:55:31.387751 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:31.387763 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:31.387806 (+ 43us) server_negotiation.cc:407] Connection header received
0928 14:55:31.389540 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:31.389564 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:31.389983 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:31.390375 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:31.391796 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:31.393115 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:31.394333 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:31.395220 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:31.395395 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:31.399678 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:31.399689 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:31.399707 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:31.399752 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:31.402780 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:31.403705 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:31.403711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:31.403721 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:31.403913 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:31.404399 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:31.404403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:31.404405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:31.404962 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:31.405192 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:31.405646 (+ 454us) server_negotiation.cc:299] Negotiation successful
0928 14:55:31.405968 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":353,"threads_started":1}
I20260928 14:55:31.407347 4000 raft_consensus.cc:482] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:31.407516 4000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:31.410413 4000 raft_consensus.cc:504] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da783e998d3441ba9625df7d853fc3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } }
I20260928 14:55:31.410959 4000 leader_election.cc:302] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da783e998d3441ba9625df7d853fc3ad; no voters:
I20260928 14:55:31.411497 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:31.411605 4000 leader_election.cc:288] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:31.411710 4005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:31.412822 4005 raft_consensus.cc:686] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [term 1 LEADER]: Becoming Leader. State: Replica: da783e998d3441ba9625df7d853fc3ad, State: Running, Role: LEADER
I20260928 14:55:31.413372 4005 consensus_queue.cc:237] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da783e998d3441ba9625df7d853fc3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } }
I20260928 14:55:31.413726 4000 sys_catalog.cc:564] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:31.418056 4006 sys_catalog.cc:455] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da783e998d3441ba9625df7d853fc3ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da783e998d3441ba9625df7d853fc3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } } }
I20260928 14:55:31.418423 4006 sys_catalog.cc:458] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:31.418881 4007 sys_catalog.cc:455] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader da783e998d3441ba9625df7d853fc3ad. Latest consensus state: current_term: 1 leader_uuid: "da783e998d3441ba9625df7d853fc3ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da783e998d3441ba9625df7d853fc3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } } }
I20260928 14:55:31.419178 4007 sys_catalog.cc:458] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:31.441615 4014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:31.453218 4014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:31.456674 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:31.428653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:45959 (local address 127.0.0.1:39181)
0928 14:55:31.428870 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:31.428877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:31.434469 (+ 5592us) server_negotiation.cc:407] Connection header received
0928 14:55:31.434657 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:31.434663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:31.434771 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:31.434870 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:31.436999 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:31.438272 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:31.440251 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:31.441038 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:31.441162 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:31.444823 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:31.444830 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:31.444836 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:31.444883 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:31.452639 (+ 7756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:31.453883 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:31.453889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:31.453892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:31.454045 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:31.454659 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:31.454665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:31.454668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:31.455094 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:31.455357 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:31.456050 (+ 693us) server_negotiation.cc:299] Negotiation successful
0928 14:55:31.456250 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260928 14:55:31.465458 4014 catalog_manager.cc:1347] Generated new cluster ID: 05eb6554dcbc4fbf9ac49ec210e43c6f
I20260928 14:55:31.465596 4014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:31.474665 4014 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:31.476029 4014 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:31.486006 4014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da783e998d3441ba9625df7d853fc3ad: Generated new TSK 0
I20260928 14:55:31.486881 4014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:31.494864 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 3933
I20260928 14:55:31.496315 3995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:31.586880 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 3933
W20260928 14:55:31.600988 3918 connection.cc:535] client connection to 127.0.0.1:39181 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39181): Connection reset by peer (error 104)
2026-09-28T14:55:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1918 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:31 dist-test-slave-vk40 krb5kdc[4033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:31 dist-test-slave-vk40 krb5kdc[4033](info): set up 2 sockets
Sep 28 14:55:31 dist-test-slave-vk40 krb5kdc[4033](info): commencing operation
krb5kdc: starting...
W20260928 14:55:32.611528 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:32 dist-test-slave-vk40 krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:32.865643 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42791
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:33.194772 4047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:33.195111 4047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:33.195250 4047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:33.202461 4047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:33.202576 4047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:33.202634 4047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:33.202723 4047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:33.202783 4047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:33.213089 4047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:33.214398 4047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:33.215723 4047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:33.227176 4052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:33.227139 4053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:33.227203 4055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:33.228372 4047 server_base.cc:1034] running on GCE node
I20260928 14:55:33.229132 4047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:33.230535 4047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:33.231752 4047 hybrid_clock.cc:648] HybridClock initialized: now 1790607333231695 us; error 49 us; skew 500 ppm
I20260928 14:55:33.232105 4047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:33 dist-test-slave-vk40 krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607333, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:33.235144 4047 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:33.236647 4047 webserver.cc:460] Webserver started at http://127.0.0.1:37399/ using document root <none> and password file <none>
I20260928 14:55:33.237320 4047 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:33.237438 4047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:33.237720 4047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:33.240715 4047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "396430adc98a4cb9a7e7bc9efb71075a"
format_stamp: "Formatted at 2026-09-28 14:55:33 on dist-test-slave-vk40"
server_key: "3d6e1bd8abec24f4d0418329964b63ed"
server_key_iv: "c593c42009812a557df5f3cad6574beb"
server_key_version: "encryptionkey@0"
I20260928 14:55:33.241466 4047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "396430adc98a4cb9a7e7bc9efb71075a"
format_stamp: "Formatted at 2026-09-28 14:55:33 on dist-test-slave-vk40"
server_key: "3d6e1bd8abec24f4d0418329964b63ed"
server_key_iv: "c593c42009812a557df5f3cad6574beb"
server_key_version: "encryptionkey@0"
I20260928 14:55:33.246564 4047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:33.250243 4062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:33.251613 4047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:55:33.251783 4047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "396430adc98a4cb9a7e7bc9efb71075a"
format_stamp: "Formatted at 2026-09-28 14:55:33 on dist-test-slave-vk40"
server_key: "3d6e1bd8abec24f4d0418329964b63ed"
server_key_iv: "c593c42009812a557df5f3cad6574beb"
server_key_version: "encryptionkey@0"
I20260928 14:55:33.251977 4047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:33.265252 4047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:33.269222 4047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:33.269476 4047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:33.287199 4047 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42791
I20260928 14:55:33.287230 4113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42791 every 8 connection(s)
I20260928 14:55:33.288650 4047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:33.289773 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 4047
I20260928 14:55:33.289955 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:33.290515 15303 external_mini_cluster.cc:1428] Setting key 174431f281c60edefa6ba903bc6149c7
I20260928 14:55:33.293279 4114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:33.303465 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:33.292421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:43963 (local address 127.0.0.1:42791)
0928 14:55:33.293136 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:33.293148 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:33.293188 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:55:33.294905 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:33.294938 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:33.295387 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:33.295808 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:33.296386 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:33.297738 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:33.299041 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:33.299745 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:33.299916 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:33.301201 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:33.301208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:33.301223 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:33.301789 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:33.301875 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:33.301903 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:33.302059 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:33.302336 (+ 277us) server_negotiation.cc:299] Negotiation successful
0928 14:55:33.302510 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":291,"threads_started":1}
I20260928 14:55:33.304216 4114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a: Bootstrap starting.
I20260928 14:55:33.307482 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:33.307890 4114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:33.308904 4114 log.cc:824] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:33.311995 4114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a: No bootstrap required, opened a new log
I20260928 14:55:33.317927 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:33.311705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:59257 (local address 127.0.0.1:42791)
0928 14:55:33.311843 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:33.311847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:33.312443 (+ 596us) server_negotiation.cc:407] Connection header received
0928 14:55:33.312642 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:33.312646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:33.312709 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:33.312792 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:33.313306 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:33.314115 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:33.315454 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:33.315957 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:33.316078 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:33.317227 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:33.317232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:33.317235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:33.317253 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:33.317311 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:33.317326 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:33.317432 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:33.317609 (+ 177us) server_negotiation.cc:299] Negotiation successful
0928 14:55:33.317663 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:55:33.320267 4114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396430adc98a4cb9a7e7bc9efb71075a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42791 } }
I20260928 14:55:33.320732 4114 raft_consensus.cc:374] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:33.320832 4114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396430adc98a4cb9a7e7bc9efb71075a, State: Initialized, Role: FOLLOWER
I20260928 14:55:33.321382 4114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396430adc98a4cb9a7e7bc9efb71075a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42791 } }
I20260928 14:55:33.321609 4114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:33.321749 4114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:33.321919 4114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:33.324695 4114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396430adc98a4cb9a7e7bc9efb71075a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42791 } }
I20260928 14:55:33.325232 4114 leader_election.cc:302] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 396430adc98a4cb9a7e7bc9efb71075a; no voters:
I20260928 14:55:33.325834 4114 leader_election.cc:288] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:33.325939 4124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:33.326589 4124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [term 1 LEADER]: Becoming Leader. State: Replica: 396430adc98a4cb9a7e7bc9efb71075a, State: Running, Role: LEADER
I20260928 14:55:33.327155 4124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396430adc98a4cb9a7e7bc9efb71075a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42791 } }
I20260928 14:55:33.327898 4114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:33.330266 4126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 396430adc98a4cb9a7e7bc9efb71075a. Latest consensus state: current_term: 1 leader_uuid: "396430adc98a4cb9a7e7bc9efb71075a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396430adc98a4cb9a7e7bc9efb71075a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42791 } } }
I20260928 14:55:33.330296 4125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "396430adc98a4cb9a7e7bc9efb71075a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396430adc98a4cb9a7e7bc9efb71075a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42791 } } }
I20260928 14:55:33.330708 4126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:33.330708 4125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:33.336032 4129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:33.341176 4129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:33.350754 4129 catalog_manager.cc:1347] Generated new cluster ID: 3945fb6887f0421fb6f75c1045aaf019
I20260928 14:55:33.350880 4129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:33.373256 4129 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:33.374571 4129 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:33.386488 4129 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 396430adc98a4cb9a7e7bc9efb71075a: Generated new TSK 0
I20260928 14:55:33.387359 4129 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:33.397827 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4047
W20260928 14:55:33.399413 4109 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.61:43963
W20260928 14:55:33.400102 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4047: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:33.400228 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 4047
W20260928 14:55:33.414186 4031 connection.cc:535] client connection to 127.0.0.1:42791 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42791): Connection reset by peer (error 104)
2026-09-28T14:55:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1813 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:33 dist-test-slave-vk40 krb5kdc[4146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:33 dist-test-slave-vk40 krb5kdc[4146](info): set up 2 sockets
Sep 28 14:55:33 dist-test-slave-vk40 krb5kdc[4146](info): commencing operation
krb5kdc: starting...
W20260928 14:55:34.411310 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:34 dist-test-slave-vk40 krb5kdc[4146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:34.666874 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:34.991833 4159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:34.992142 4159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:34.992304 4159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:34.999663 4159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:34.999799 4159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:34.999881 4159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:34.999963 4159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:35.000025 4159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:35.010083 4159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:35.011371 4159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:35.012723 4159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:35.024272 4164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:35.024366 4167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:35.024338 4165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:35.025028 4159 server_base.cc:1034] running on GCE node
I20260928 14:55:35.025837 4159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:35.027333 4159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:35.028532 4159 hybrid_clock.cc:648] HybridClock initialized: now 1790607335028487 us; error 43 us; skew 500 ppm
I20260928 14:55:35.028903 4159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:35 dist-test-slave-vk40 krb5kdc[4146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607335, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:35.035972 4159 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:35.037647 4159 webserver.cc:460] Webserver started at http://127.0.0.1:45531/ using document root <none> and password file <none>
I20260928 14:55:35.038340 4159 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:35.038461 4159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:35.038767 4159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:35.041949 4159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "2d62066f7be6403d92f785d393807070"
format_stamp: "Formatted at 2026-09-28 14:55:35 on dist-test-slave-vk40"
server_key: "49caa70fdb04ff02fa2470013d49539d"
server_key_iv: "6a667add87084571997222e37188f54c"
server_key_version: "encryptionkey@0"
I20260928 14:55:35.042717 4159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "2d62066f7be6403d92f785d393807070"
format_stamp: "Formatted at 2026-09-28 14:55:35 on dist-test-slave-vk40"
server_key: "49caa70fdb04ff02fa2470013d49539d"
server_key_iv: "6a667add87084571997222e37188f54c"
server_key_version: "encryptionkey@0"
I20260928 14:55:35.048563 4159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260928 14:55:35.054895 4174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:35.056437 4159 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:55:35.056597 4159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "2d62066f7be6403d92f785d393807070"
format_stamp: "Formatted at 2026-09-28 14:55:35 on dist-test-slave-vk40"
server_key: "49caa70fdb04ff02fa2470013d49539d"
server_key_iv: "6a667add87084571997222e37188f54c"
server_key_version: "encryptionkey@0"
I20260928 14:55:35.056798 4159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:35.062687 4159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:35.066864 4159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:35.067135 4159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:35.084466 4159 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44523
I20260928 14:55:35.084502 4225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44523 every 8 connection(s)
I20260928 14:55:35.085826 4159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:35.089599 4226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:35.090608 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 4159
I20260928 14:55:35.090798 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:35.091321 15303 external_mini_cluster.cc:1428] Setting key 63e08d25f12ed528d00e5a2b176379b7
I20260928 14:55:35.099126 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:35.093167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:46849 (local address 127.0.0.1:44523)
0928 14:55:35.093942 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:35.093954 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:35.093999 (+ 45us) server_negotiation.cc:407] Connection header received
0928 14:55:35.095724 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:35.095752 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:35.096136 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:35.096517 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:35.097534 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:35.097540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:35.097555 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:35.097570 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:35.097630 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:35.097653 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:35.097810 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:35.097936 (+ 126us) server_negotiation.cc:299] Negotiation successful
0928 14:55:35.098116 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":326,"threads_started":1}
I20260928 14:55:35.103052 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:35.103173 4226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070: Bootstrap starting.
I20260928 14:55:35.107036 4226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:35.108361 4226 log.cc:824] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:35.110695 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:35.108168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:35055 (local address 127.0.0.1:44523)
0928 14:55:35.108385 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:35.108391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:35.108780 (+ 389us) server_negotiation.cc:407] Connection header received
0928 14:55:35.108952 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:35.108961 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:35.109023 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:35.109113 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:35.110090 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:35.110095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:35.110100 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:35.110135 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:35.110184 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:35.110205 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:35.110319 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:35.110362 (+ 43us) server_negotiation.cc:299] Negotiation successful
0928 14:55:35.110424 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:55:35.111806 4226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070: No bootstrap required, opened a new log
I20260928 14:55:35.120450 4226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d62066f7be6403d92f785d393807070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44523 } }
I20260928 14:55:35.120885 4226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:35.120986 4226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62066f7be6403d92f785d393807070, State: Initialized, Role: FOLLOWER
I20260928 14:55:35.121552 4226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d62066f7be6403d92f785d393807070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44523 } }
I20260928 14:55:35.121783 4226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:35.121920 4226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:35.122066 4226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:35.124519 4226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d62066f7be6403d92f785d393807070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44523 } }
I20260928 14:55:35.125012 4226 leader_election.cc:302] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d62066f7be6403d92f785d393807070; no voters:
I20260928 14:55:35.125572 4226 leader_election.cc:288] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:35.125703 4236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:35.126329 4236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [term 1 LEADER]: Becoming Leader. State: Replica: 2d62066f7be6403d92f785d393807070, State: Running, Role: LEADER
I20260928 14:55:35.126865 4236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d62066f7be6403d92f785d393807070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44523 } }
I20260928 14:55:35.127584 4226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:35.129982 4238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d62066f7be6403d92f785d393807070. Latest consensus state: current_term: 1 leader_uuid: "2d62066f7be6403d92f785d393807070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d62066f7be6403d92f785d393807070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44523 } } }
I20260928 14:55:35.130028 4237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d62066f7be6403d92f785d393807070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d62066f7be6403d92f785d393807070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44523 } } }
I20260928 14:55:35.130451 4238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:35.130476 4237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:35.132741 4243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:35.137832 4243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:35.146926 4243 catalog_manager.cc:1347] Generated new cluster ID: 0a5c596bc6124b6096080a3c582ebfe2
I20260928 14:55:35.147044 4243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:35.160806 4243 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:35.161254 4243 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:35.167524 4243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d62066f7be6403d92f785d393807070: Generated new TSK 0
I20260928 14:55:35.168375 4243 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:35.178611 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4159
W20260928 14:55:35.179951 4221 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.61:46849
W20260928 14:55:35.180756 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4159: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:35.180899 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 4159
2026-09-28T14:55:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1780 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:35 dist-test-slave-vk40 krb5kdc[4257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:35 dist-test-slave-vk40 krb5kdc[4257](info): set up 2 sockets
Sep 28 14:55:35 dist-test-slave-vk40 krb5kdc[4257](info): commencing operation
krb5kdc: starting...
W20260928 14:55:36.211408 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:36 dist-test-slave-vk40 krb5kdc[4257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:36.466980 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36361
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:36.794831 4271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:36.795121 4271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:36.795213 4271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:36.802402 4271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:36.802508 4271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:36.802560 4271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:36.802626 4271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:36.802676 4271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:36.812562 4271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36361
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:36.813854 4271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:36.815138 4271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:36.826599 4276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:36.826551 4279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:36.826632 4277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:36.827718 4271 server_base.cc:1034] running on GCE node
I20260928 14:55:36.828543 4271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:36.830010 4271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:36.831238 4271 hybrid_clock.cc:648] HybridClock initialized: now 1790607336831191 us; error 41 us; skew 500 ppm
I20260928 14:55:36.831570 4271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:36 dist-test-slave-vk40 krb5kdc[4257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607336, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:36.834554 4271 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:36.836004 4271 webserver.cc:460] Webserver started at http://127.0.0.1:40073/ using document root <none> and password file <none>
I20260928 14:55:36.836714 4271 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:36.836817 4271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:36.837067 4271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:36.840018 4271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "5934b6f90d414244ad72d3e0421b8c32"
format_stamp: "Formatted at 2026-09-28 14:55:36 on dist-test-slave-vk40"
server_key: "4146c0c20182255e2cd828ddf859c9d5"
server_key_iv: "7d1dcd72ea06c6b876195d4bc2a87323"
server_key_version: "encryptionkey@0"
I20260928 14:55:36.840827 4271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "5934b6f90d414244ad72d3e0421b8c32"
format_stamp: "Formatted at 2026-09-28 14:55:36 on dist-test-slave-vk40"
server_key: "4146c0c20182255e2cd828ddf859c9d5"
server_key_iv: "7d1dcd72ea06c6b876195d4bc2a87323"
server_key_version: "encryptionkey@0"
I20260928 14:55:36.845896 4271 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:55:36.849309 4286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:36.850742 4271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:55:36.850916 4271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "5934b6f90d414244ad72d3e0421b8c32"
format_stamp: "Formatted at 2026-09-28 14:55:36 on dist-test-slave-vk40"
server_key: "4146c0c20182255e2cd828ddf859c9d5"
server_key_iv: "7d1dcd72ea06c6b876195d4bc2a87323"
server_key_version: "encryptionkey@0"
I20260928 14:55:36.851096 4271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:36.870151 4271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:36.874449 4271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:36.874706 4271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:36.893424 4271 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36361
I20260928 14:55:36.893446 4337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36361 every 8 connection(s)
I20260928 14:55:36.894826 4271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:36.898976 4338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:36.901640 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 4271
I20260928 14:55:36.901852 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:36.902264 15303 external_mini_cluster.cc:1428] Setting key 6b6ceae82ba80f7406f202f7d273e3ff
I20260928 14:55:36.911563 4338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32: Bootstrap starting.
I20260928 14:55:36.915453 4338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:36.916839 4338 log.cc:824] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32: Log is configured to *not* fsync() on all Append() calls
Sep 28 14:55:36 dist-test-slave-vk40 krb5kdc[4257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:55:36.920318 4338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32: No bootstrap required, opened a new log
I20260928 14:55:36.927403 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:36.904083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:58837 (local address 127.0.0.1:36361)
0928 14:55:36.904927 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:36.904940 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:36.904983 (+ 43us) server_negotiation.cc:407] Connection header received
0928 14:55:36.906699 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:36.906724 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:36.907164 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:36.907649 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:36.909981 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:36.911375 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:36.913154 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:36.913968 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:36.914110 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:36.919328 (+ 5218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:36.919337 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:36.919352 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:36.919393 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:36.922548 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:36.923659 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:36.923667 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:36.923673 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:36.923886 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:36.924423 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:36.924427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:36.924429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:36.925033 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:36.925334 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:36.925868 (+ 534us) server_negotiation.cc:299] Negotiation successful
0928 14:55:36.926261 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":377,"threads_started":1}
I20260928 14:55:36.929733 4338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5934b6f90d414244ad72d3e0421b8c32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } }
I20260928 14:55:36.930143 4338 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:36.930262 4338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5934b6f90d414244ad72d3e0421b8c32, State: Initialized, Role: FOLLOWER
I20260928 14:55:36.930864 4338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5934b6f90d414244ad72d3e0421b8c32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } }
I20260928 14:55:36.931141 4338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:36.931296 4338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:36.931470 4338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:36.931843 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:36.933772 4338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5934b6f90d414244ad72d3e0421b8c32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } }
I20260928 14:55:36.934258 4338 leader_election.cc:302] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5934b6f90d414244ad72d3e0421b8c32; no voters:
I20260928 14:55:36.935797 4338 leader_election.cc:288] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:36.935949 4346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:36.937254 4346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [term 1 LEADER]: Becoming Leader. State: Replica: 5934b6f90d414244ad72d3e0421b8c32, State: Running, Role: LEADER
I20260928 14:55:36.937922 4346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5934b6f90d414244ad72d3e0421b8c32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } }
I20260928 14:55:36.938335 4338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:36.940810 4348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5934b6f90d414244ad72d3e0421b8c32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5934b6f90d414244ad72d3e0421b8c32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } } }
I20260928 14:55:36.940845 4350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5934b6f90d414244ad72d3e0421b8c32. Latest consensus state: current_term: 1 leader_uuid: "5934b6f90d414244ad72d3e0421b8c32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5934b6f90d414244ad72d3e0421b8c32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } } }
I20260928 14:55:36.941196 4348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:36.941197 4350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:36.942749 4354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:36.947780 4354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:36.959012 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:36.936143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:52653 (local address 127.0.0.1:36361)
0928 14:55:36.936361 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:36.936368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:36.937917 (+ 1549us) server_negotiation.cc:407] Connection header received
0928 14:55:36.938253 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:36.938259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:36.938348 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:36.938447 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:36.942535 (+ 4088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:36.943863 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:36.945976 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:36.946770 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:36.946879 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:36.950584 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:36.950591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:36.950597 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:36.950664 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:36.954747 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:36.956023 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:36.956030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:36.956035 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:36.956205 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:36.956849 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:36.956855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:36.956859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:36.957329 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:36.957650 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:36.958369 (+ 719us) server_negotiation.cc:299] Negotiation successful
0928 14:55:36.958587 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:55:36.965758 4354 catalog_manager.cc:1347] Generated new cluster ID: 393fe9ed4b704ad1aaef3d4e2188d001
I20260928 14:55:36.965879 4354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:36.979288 4354 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:36.980652 4354 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:36.988118 4354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5934b6f90d414244ad72d3e0421b8c32: Generated new TSK 0
I20260928 14:55:36.989008 4354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:37.000120 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4271
I20260928 14:55:37.001276 4333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:37.093911 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 4271
W20260928 14:55:37.108266 4255 connection.cc:535] client connection to 127.0.0.1:36361 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36361): Connection reset by peer (error 104)
2026-09-28T14:55:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1913 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:37 dist-test-slave-vk40 krb5kdc[4371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:37 dist-test-slave-vk40 krb5kdc[4371](info): set up 2 sockets
Sep 28 14:55:37 dist-test-slave-vk40 krb5kdc[4371](info): commencing operation
krb5kdc: starting...
W20260928 14:55:38.107111 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:38 dist-test-slave-vk40 krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:38.358855 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40249
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:38.684161 4384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:38.684514 4384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:38.684634 4384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:38.691952 4384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:38.692072 4384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:38.692118 4384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:38.692183 4384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:38.692233 4384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:38.702370 4384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40249
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:38.703658 4384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:38.705005 4384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:38.716790 4390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:38.716838 4392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:38.717298 4389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:38.717639 4384 server_base.cc:1034] running on GCE node
I20260928 14:55:38.718436 4384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:38.719937 4384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:38.721148 4384 hybrid_clock.cc:648] HybridClock initialized: now 1790607338721104 us; error 28 us; skew 500 ppm
I20260928 14:55:38.721505 4384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:38 dist-test-slave-vk40 krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607338, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:38.724587 4384 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:38.726035 4384 webserver.cc:460] Webserver started at http://127.0.0.1:40843/ using document root <none> and password file <none>
I20260928 14:55:38.726703 4384 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:38.726818 4384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:38.727102 4384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:38.730106 4384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "cb773a4115a746b086c9d327820320fe"
format_stamp: "Formatted at 2026-09-28 14:55:38 on dist-test-slave-vk40"
server_key: "b7b5471e928502ca5fe025dadb96eaee"
server_key_iv: "4dd925f21353d583e0bd104758c0c0b4"
server_key_version: "encryptionkey@0"
I20260928 14:55:38.730865 4384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "cb773a4115a746b086c9d327820320fe"
format_stamp: "Formatted at 2026-09-28 14:55:38 on dist-test-slave-vk40"
server_key: "b7b5471e928502ca5fe025dadb96eaee"
server_key_iv: "4dd925f21353d583e0bd104758c0c0b4"
server_key_version: "encryptionkey@0"
I20260928 14:55:38.736306 4384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 14:55:38.739868 4399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:38.741258 4384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:55:38.741423 4384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "cb773a4115a746b086c9d327820320fe"
format_stamp: "Formatted at 2026-09-28 14:55:38 on dist-test-slave-vk40"
server_key: "b7b5471e928502ca5fe025dadb96eaee"
server_key_iv: "4dd925f21353d583e0bd104758c0c0b4"
server_key_version: "encryptionkey@0"
I20260928 14:55:38.741611 4384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:38.777720 4384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:38.782187 4384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:38.782474 4384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:38.800565 4384 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40249
I20260928 14:55:38.800589 4451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40249 every 8 connection(s)
I20260928 14:55:38.802112 4384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:38.803397 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 4384
I20260928 14:55:38.803570 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:38.804066 15303 external_mini_cluster.cc:1428] Setting key 9d9f6d34b8af28e075ca0ff0f1bcc0c4
I20260928 14:55:38.806653 4452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:38.817433 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:38.805856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:49809 (local address 127.0.0.1:40249)
0928 14:55:38.806630 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:38.806646 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:38.806689 (+ 43us) server_negotiation.cc:407] Connection header received
0928 14:55:38.808410 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:38.808434 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:38.808835 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:38.809208 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:38.809780 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:38.811104 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:38.812962 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:38.813657 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:38.813797 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:38.815023 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:38.815030 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:38.815048 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:38.815606 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:38.815691 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:38.815717 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:38.815840 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:38.816130 (+ 290us) server_negotiation.cc:299] Negotiation successful
0928 14:55:38.816365 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":327,"threads_started":1}
I20260928 14:55:38.819242 4452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe: Bootstrap starting.
I20260928 14:55:38.821470 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:38.822605 4452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:38.823621 4452 log.cc:824] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:38.826646 4452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe: No bootstrap required, opened a new log
I20260928 14:55:38.832154 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:38.825794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:41187 (local address 127.0.0.1:40249)
0928 14:55:38.825976 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:38.825981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:38.826360 (+ 379us) server_negotiation.cc:407] Connection header received
0928 14:55:38.826518 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:38.826523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:38.826601 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:38.826732 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:38.827249 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:38.828368 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:38.829606 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:38.830293 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:38.830418 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:38.831390 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:38.831396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:38.831401 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:38.831424 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:38.831475 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:38.831495 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:38.831613 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:38.831733 (+ 120us) server_negotiation.cc:299] Negotiation successful
0928 14:55:38.831797 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:55:38.835640 4452 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb773a4115a746b086c9d327820320fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } }
I20260928 14:55:38.836028 4452 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:38.836104 4452 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb773a4115a746b086c9d327820320fe, State: Initialized, Role: FOLLOWER
I20260928 14:55:38.836659 4452 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb773a4115a746b086c9d327820320fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } }
I20260928 14:55:38.836881 4452 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:38.836978 4452 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:38.837100 4452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:38.839257 4452 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb773a4115a746b086c9d327820320fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } }
I20260928 14:55:38.839803 4452 leader_election.cc:302] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb773a4115a746b086c9d327820320fe; no voters:
I20260928 14:55:38.840474 4452 leader_election.cc:288] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:38.840607 4462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:38.841220 4462 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [term 1 LEADER]: Becoming Leader. State: Replica: cb773a4115a746b086c9d327820320fe, State: Running, Role: LEADER
I20260928 14:55:38.841764 4462 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb773a4115a746b086c9d327820320fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } }
I20260928 14:55:38.842478 4452 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:38.844347 4464 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb773a4115a746b086c9d327820320fe. Latest consensus state: current_term: 1 leader_uuid: "cb773a4115a746b086c9d327820320fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb773a4115a746b086c9d327820320fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } } }
I20260928 14:55:38.844373 4463 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb773a4115a746b086c9d327820320fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb773a4115a746b086c9d327820320fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } } }
I20260928 14:55:38.844789 4463 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:38.844786 4464 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:38.848156 4467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:38.853315 4467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:38.862391 4467 catalog_manager.cc:1347] Generated new cluster ID: e3a1f78f2efb4024ac8ff39bd955e9aa
I20260928 14:55:38.862502 4467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:38.896095 4467 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:38.897471 4467 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:38.904397 4467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb773a4115a746b086c9d327820320fe: Generated new TSK 0
I20260928 14:55:38.905498 4467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:38.921536 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4384
W20260928 14:55:38.922747 4447 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.61:49809
W20260928 14:55:38.923489 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4384: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:38.923645 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 4384
W20260928 14:55:38.937877 4369 connection.cc:535] client connection to 127.0.0.1:40249 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40249): Connection reset by peer (error 104)
2026-09-28T14:55:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1829 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:39 dist-test-slave-vk40 krb5kdc[4483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:39 dist-test-slave-vk40 krb5kdc[4483](info): set up 2 sockets
Sep 28 14:55:39 dist-test-slave-vk40 krb5kdc[4483](info): commencing operation
krb5kdc: starting...
W20260928 14:55:39.955330 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:40 dist-test-slave-vk40 krb5kdc[4483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:55:40.213199 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43577
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:40.541234 4496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:40.541558 4496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:40.541702 4496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:40.549067 4496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:40.549204 4496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:40.549285 4496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:40.549363 4496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:40.549425 4496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:40.559396 4496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43577
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:40.560727 4496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:40.562037 4496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:40.573659 4502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:40.573648 4501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:40.573709 4504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:40.574621 4496 server_base.cc:1034] running on GCE node
I20260928 14:55:40.575440 4496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:40.576982 4496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:40.578198 4496 hybrid_clock.cc:648] HybridClock initialized: now 1790607340578147 us; error 44 us; skew 500 ppm
I20260928 14:55:40.578549 4496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:40 dist-test-slave-vk40 krb5kdc[4483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607340, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:40.586254 4496 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:55:40.587947 4496 webserver.cc:460] Webserver started at http://127.0.0.1:42211/ using document root <none> and password file <none>
I20260928 14:55:40.588697 4496 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:40.588828 4496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:40.589121 4496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:40.592155 4496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "5cc93ddb3be14b5e8252d6e0bdbb87f5"
format_stamp: "Formatted at 2026-09-28 14:55:40 on dist-test-slave-vk40"
server_key: "502fc620e05c6ee2bc08bf5176925d1c"
server_key_iv: "6171d27c79184908d30008c71910b6c6"
server_key_version: "encryptionkey@0"
I20260928 14:55:40.592970 4496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "5cc93ddb3be14b5e8252d6e0bdbb87f5"
format_stamp: "Formatted at 2026-09-28 14:55:40 on dist-test-slave-vk40"
server_key: "502fc620e05c6ee2bc08bf5176925d1c"
server_key_iv: "6171d27c79184908d30008c71910b6c6"
server_key_version: "encryptionkey@0"
I20260928 14:55:40.598161 4496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:55:40.601789 4511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:40.603153 4496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:55:40.603322 4496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "5cc93ddb3be14b5e8252d6e0bdbb87f5"
format_stamp: "Formatted at 2026-09-28 14:55:40 on dist-test-slave-vk40"
server_key: "502fc620e05c6ee2bc08bf5176925d1c"
server_key_iv: "6171d27c79184908d30008c71910b6c6"
server_key_version: "encryptionkey@0"
I20260928 14:55:40.603515 4496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:40.609364 4496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:40.613745 4496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:40.614045 4496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:40.631353 4496 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43577
I20260928 14:55:40.631403 4562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43577 every 8 connection(s)
I20260928 14:55:40.632781 4496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:40.636402 4563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:40.637305 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 4496
I20260928 14:55:40.637452 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:40.637877 15303 external_mini_cluster.cc:1428] Setting key 7a05ec0aca7644c89622957b5cb87736
I20260928 14:55:40.645982 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:40.639888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:60117 (local address 127.0.0.1:43577)
0928 14:55:40.640666 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:40.640680 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:40.640722 (+ 42us) server_negotiation.cc:407] Connection header received
0928 14:55:40.642653 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:40.642676 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:40.643062 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:40.643483 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:40.644436 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:40.644442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:40.644455 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:40.644469 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:40.644527 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:40.644546 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:40.644741 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:40.644881 (+ 140us) server_negotiation.cc:299] Negotiation successful
0928 14:55:40.645067 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":344,"threads_started":1}
I20260928 14:55:40.649731 4563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5: Bootstrap starting.
I20260928 14:55:40.649776 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:40.652684 4563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:40.653776 4563 log.cc:824] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:40.656239 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:40.654009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:60427 (local address 127.0.0.1:43577)
0928 14:55:40.654190 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:40.654196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:40.654502 (+ 306us) server_negotiation.cc:407] Connection header received
0928 14:55:40.654671 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:40.654677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:40.654734 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:40.654829 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:40.655669 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:40.655675 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:40.655679 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:40.655697 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:40.655734 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:40.655747 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:40.655854 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:40.655929 (+ 75us) server_negotiation.cc:299] Negotiation successful
0928 14:55:40.655988 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 14:55:40.656879 4563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5: No bootstrap required, opened a new log
I20260928 14:55:40.664196 4563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc93ddb3be14b5e8252d6e0bdbb87f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43577 } }
I20260928 14:55:40.664629 4563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:40.664723 4563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc93ddb3be14b5e8252d6e0bdbb87f5, State: Initialized, Role: FOLLOWER
I20260928 14:55:40.665369 4563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc93ddb3be14b5e8252d6e0bdbb87f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43577 } }
I20260928 14:55:40.665652 4563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:40.665787 4563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:40.665931 4563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:40.668094 4563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc93ddb3be14b5e8252d6e0bdbb87f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43577 } }
I20260928 14:55:40.668615 4563 leader_election.cc:302] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cc93ddb3be14b5e8252d6e0bdbb87f5; no voters:
I20260928 14:55:40.669245 4563 leader_election.cc:288] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:40.669384 4573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:40.671339 4563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:40.672727 4573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [term 1 LEADER]: Becoming Leader. State: Replica: 5cc93ddb3be14b5e8252d6e0bdbb87f5, State: Running, Role: LEADER
I20260928 14:55:40.673331 4573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc93ddb3be14b5e8252d6e0bdbb87f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43577 } }
I20260928 14:55:40.676224 4576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cc93ddb3be14b5e8252d6e0bdbb87f5. Latest consensus state: current_term: 1 leader_uuid: "5cc93ddb3be14b5e8252d6e0bdbb87f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc93ddb3be14b5e8252d6e0bdbb87f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43577 } } }
I20260928 14:55:40.677533 4576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:40.677001 4574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cc93ddb3be14b5e8252d6e0bdbb87f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc93ddb3be14b5e8252d6e0bdbb87f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43577 } } }
I20260928 14:55:40.677779 4574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5 [sys.catalog]: This master's current role is: LEADER
W20260928 14:55:40.681954 4588 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:55:40.682444 4588 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:55:40.683410 4589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:40.685211 4589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:40.693259 4589 catalog_manager.cc:1347] Generated new cluster ID: 823ac943438b44d5ab6441fdd18d5e97
I20260928 14:55:40.693395 4589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:40.710399 4589 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:40.710875 4589 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:40.722030 4589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cc93ddb3be14b5e8252d6e0bdbb87f5: Generated new TSK 0
I20260928 14:55:40.723119 4589 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:40.728782 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4496
W20260928 14:55:40.730010 4558 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.61:60117
W20260928 14:55:40.730734 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4496: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:40.730875 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 4496
2026-09-28T14:55:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1807 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:40 dist-test-slave-vk40 krb5kdc[4595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:40 dist-test-slave-vk40 krb5kdc[4595](info): set up 2 sockets
Sep 28 14:55:40 dist-test-slave-vk40 krb5kdc[4595](info): commencing operation
krb5kdc: starting...
W20260928 14:55:41.767210 15303 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:41 dist-test-slave-vk40 krb5kdc[4595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:55:42.035460 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:33741
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:41519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:33741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:42.361837 4608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:42.362150 4608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:42.362289 4608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:42.369789 4608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:42.369943 4608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:42.370025 4608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:42.370112 4608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:42.370172 4608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:42.380314 4608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:41519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:33741
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:33741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:42.381654 4608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:42.382983 4608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:42.394917 4613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:42.394855 4616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:42.394882 4614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:42.396194 4608 server_base.cc:1034] running on GCE node
I20260928 14:55:42.397042 4608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:42.398515 4608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:42.399732 4608 hybrid_clock.cc:648] HybridClock initialized: now 1790607342399682 us; error 44 us; skew 500 ppm
I20260928 14:55:42.400098 4608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:42 dist-test-slave-vk40 krb5kdc[4595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607342, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:42.403168 4608 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:55:42.404673 4608 webserver.cc:460] Webserver started at http://127.14.241.254:32837/ using document root <none> and password file <none>
I20260928 14:55:42.405365 4608 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:42.405483 4608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:42.405762 4608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:42.408833 4608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "77e474f06634421a98bd01c341ed5dda"
format_stamp: "Formatted at 2026-09-28 14:55:42 on dist-test-slave-vk40"
server_key: "93cae8b85be3c61145e757e7b01269ec"
server_key_iv: "1e4d758f0fd4d345f71c99a45cf20240"
server_key_version: "encryptionkey@0"
I20260928 14:55:42.409598 4608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "77e474f06634421a98bd01c341ed5dda"
format_stamp: "Formatted at 2026-09-28 14:55:42 on dist-test-slave-vk40"
server_key: "93cae8b85be3c61145e757e7b01269ec"
server_key_iv: "1e4d758f0fd4d345f71c99a45cf20240"
server_key_version: "encryptionkey@0"
I20260928 14:55:42.414920 4608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:42.418542 4623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:42.419904 4608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 14:55:42.420081 4608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "77e474f06634421a98bd01c341ed5dda"
format_stamp: "Formatted at 2026-09-28 14:55:42 on dist-test-slave-vk40"
server_key: "93cae8b85be3c61145e757e7b01269ec"
server_key_iv: "1e4d758f0fd4d345f71c99a45cf20240"
server_key_version: "encryptionkey@0"
I20260928 14:55:42.420310 4608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:42.437278 4608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:42.441646 4608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:42.441922 4608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:42.459625 4608 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:33741
I20260928 14:55:42.459643 4675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:33741 every 8 connection(s)
I20260928 14:55:42.461014 4608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:42.464831 4676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:42.469328 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 4608
I20260928 14:55:42.469523 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:42.470046 15303 external_mini_cluster.cc:1428] Setting key b9e0c29271c9ec3b6fcd7dcd9a3843c6
I20260928 14:55:42.476220 4676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda: Bootstrap starting.
I20260928 14:55:42.479310 4676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:42.480351 4676 log.cc:824] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:42.482995 4676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda: No bootstrap required, opened a new log
Sep 28 14:55:42 dist-test-slave-vk40 krb5kdc[4595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:42.491854 4676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e474f06634421a98bd01c341ed5dda" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33741 } }
I20260928 14:55:42.492228 4676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:42.492342 4676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e474f06634421a98bd01c341ed5dda, State: Initialized, Role: FOLLOWER
I20260928 14:55:42.492479 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:42.471957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57651 (local address 127.14.241.254:33741)
0928 14:55:42.472746 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:42.472758 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:42.472801 (+ 43us) server_negotiation.cc:407] Connection header received
0928 14:55:42.474914 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:42.474939 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:42.475376 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:42.475751 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:42.477267 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:42.478609 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:42.479913 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:42.480678 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:42.480862 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:42.485134 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:42.485141 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:42.485159 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:42.485194 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:42.488183 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:42.489092 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:42.489099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:42.489105 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:42.489285 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:42.489743 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:42.489750 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:42.489752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:42.490334 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:42.490580 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:42.491042 (+ 462us) server_negotiation.cc:299] Negotiation successful
0928 14:55:42.491362 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":356,"threads_started":1}
I20260928 14:55:42.492949 4676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e474f06634421a98bd01c341ed5dda" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33741 } }
I20260928 14:55:42.493172 4676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:42.493341 4676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:42.493492 4676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:42.496459 4676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e474f06634421a98bd01c341ed5dda" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33741 } }
I20260928 14:55:42.496567 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:42.497012 4676 leader_election.cc:302] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77e474f06634421a98bd01c341ed5dda; no voters:
I20260928 14:55:42.497609 4676 leader_election.cc:288] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:42.497725 4681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:42.498813 4681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [term 1 LEADER]: Becoming Leader. State: Replica: 77e474f06634421a98bd01c341ed5dda, State: Running, Role: LEADER
I20260928 14:55:42.499352 4681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e474f06634421a98bd01c341ed5dda" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33741 } }
I20260928 14:55:42.499791 4676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:42.507759 4683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77e474f06634421a98bd01c341ed5dda. Latest consensus state: current_term: 1 leader_uuid: "77e474f06634421a98bd01c341ed5dda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e474f06634421a98bd01c341ed5dda" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33741 } } }
I20260928 14:55:42.508162 4683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:42.508643 4682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77e474f06634421a98bd01c341ed5dda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77e474f06634421a98bd01c341ed5dda" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 33741 } } }
I20260928 14:55:42.508908 4682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:42.512022 4702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:42.517639 4702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:42.520849 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:42.502510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56033 (local address 127.14.241.254:33741)
0928 14:55:42.502694 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:42.502704 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:42.504749 (+ 2045us) server_negotiation.cc:407] Connection header received
0928 14:55:42.504937 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:42.504943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:42.505036 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:42.505157 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:42.507411 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:42.508801 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:42.510919 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:42.511615 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:42.511721 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:42.515032 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:42.515037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:42.515041 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:42.515071 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:42.517968 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:42.518751 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:42.518756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:42.518758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:42.518881 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:42.519343 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:42.519347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:42.519349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:42.519654 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:42.519890 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:42.520302 (+ 412us) server_negotiation.cc:299] Negotiation successful
0928 14:55:42.520503 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:55:42.527321 4702 catalog_manager.cc:1347] Generated new cluster ID: 93dc7ad40ca5421bab3591f9e639939a
I20260928 14:55:42.527446 4702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:42.558198 4702 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:42.560035 4702 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:42.574571 4702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77e474f06634421a98bd01c341ed5dda: Generated new TSK 0
I20260928 14:55:42.575685 4702 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:42.588694 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4608
I20260928 14:55:42.590602 4671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:42.685221 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 4608
W20260928 14:55:42.701023 4593 connection.cc:535] client connection to 127.14.241.254:33741 recv error: Network error: failed to read from TLS socket (remote: 127.14.241.254:33741): Connection reset by peer (error 104)
2026-09-28T14:55:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1956 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:42 dist-test-slave-vk40 krb5kdc[4709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:42 dist-test-slave-vk40 krb5kdc[4709](info): set up 2 sockets
Sep 28 14:55:42 dist-test-slave-vk40 krb5kdc[4709](info): commencing operation
krb5kdc: starting...
W20260928 14:55:43.703440 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:43 dist-test-slave-vk40 krb5kdc[4709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:55:43.948748 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41033
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:38113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:41033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:44.271579 4722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:44.271878 4722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:44.272010 4722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:44.279282 4722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:44.279395 4722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:44.279469 4722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:44.279551 4722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:44.279610 4722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:44.289516 4722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:38113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:41033
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:41033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:44.290812 4722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:44.292112 4722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:44.303363 4727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:44.303485 4728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:44.303517 4730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:44.304437 4722 server_base.cc:1034] running on GCE node
I20260928 14:55:44.305205 4722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:44.306593 4722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:44.307801 4722 hybrid_clock.cc:648] HybridClock initialized: now 1790607344307754 us; error 37 us; skew 500 ppm
I20260928 14:55:44.308118 4722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:44 dist-test-slave-vk40 krb5kdc[4709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607344, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:44.311055 4722 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:55:44.312497 4722 webserver.cc:460] Webserver started at http://127.14.241.254:35249/ using document root <none> and password file <none>
I20260928 14:55:44.313133 4722 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:44.313247 4722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:44.313530 4722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:44.316450 4722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "1fe6528b287a4ac1ab4f94fe65db0ad7"
format_stamp: "Formatted at 2026-09-28 14:55:44 on dist-test-slave-vk40"
server_key: "6d7bcbb71ae3e076e027ae0a8e045dfe"
server_key_iv: "2c263f72464ea533331e2c930946d3c0"
server_key_version: "encryptionkey@0"
I20260928 14:55:44.317157 4722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "1fe6528b287a4ac1ab4f94fe65db0ad7"
format_stamp: "Formatted at 2026-09-28 14:55:44 on dist-test-slave-vk40"
server_key: "6d7bcbb71ae3e076e027ae0a8e045dfe"
server_key_iv: "2c263f72464ea533331e2c930946d3c0"
server_key_version: "encryptionkey@0"
I20260928 14:55:44.322147 4722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:55:44.325558 4737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:44.326871 4722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:55:44.327028 4722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "1fe6528b287a4ac1ab4f94fe65db0ad7"
format_stamp: "Formatted at 2026-09-28 14:55:44 on dist-test-slave-vk40"
server_key: "6d7bcbb71ae3e076e027ae0a8e045dfe"
server_key_iv: "2c263f72464ea533331e2c930946d3c0"
server_key_version: "encryptionkey@0"
I20260928 14:55:44.327210 4722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:44.338900 4722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:44.342566 4722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:44.342824 4722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:44.360213 4722 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:41033
I20260928 14:55:44.360251 4789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:41033 every 8 connection(s)
I20260928 14:55:44.361577 4722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:44.365360 4790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:44.370478 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 4722
I20260928 14:55:44.370656 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:44.371071 15303 external_mini_cluster.cc:1428] Setting key 4751e19d30c9ca5cca0d8420a42e77d4
I20260928 14:55:44.377453 4790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7: Bootstrap starting.
I20260928 14:55:44.381081 4790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:44.382383 4790 log.cc:824] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:44.385675 4790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7: No bootstrap required, opened a new log
I20260928 14:55:44.385946 4793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:44.372922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50267 (local address 127.14.241.254:41033)
0928 14:55:44.373731 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:44.373746 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:44.373804 (+ 58us) server_negotiation.cc:407] Connection header received
0928 14:55:44.376191 (+ 2387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:44.376220 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:44.376768 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:44.377209 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:44.377725 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:44.379650 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:44.381013 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:44.382030 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:44.382227 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:44.383365 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:44.383372 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:44.383386 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:44.384018 (+ 632us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:44.384094 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:44.384128 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:44.384314 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:44.384548 (+ 234us) server_negotiation.cc:299] Negotiation successful
0928 14:55:44.384716 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":313,"threads_started":1}
I20260928 14:55:44.389752 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:44.394129 4790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe6528b287a4ac1ab4f94fe65db0ad7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41033 } }
I20260928 14:55:44.394542 4790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:44.394661 4790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe6528b287a4ac1ab4f94fe65db0ad7, State: Initialized, Role: FOLLOWER
I20260928 14:55:44.395329 4790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe6528b287a4ac1ab4f94fe65db0ad7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41033 } }
I20260928 14:55:44.395622 4790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:44.395773 4790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:44.395951 4790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:44.398622 4790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe6528b287a4ac1ab4f94fe65db0ad7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41033 } }
I20260928 14:55:44.399184 4790 leader_election.cc:302] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fe6528b287a4ac1ab4f94fe65db0ad7; no voters:
I20260928 14:55:44.399878 4790 leader_election.cc:288] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:44.400043 4793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:44.393860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56013 (local address 127.14.241.254:41033)
0928 14:55:44.394025 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:44.394030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:44.394312 (+ 282us) server_negotiation.cc:407] Connection header received
0928 14:55:44.394457 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:44.394462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:44.394540 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:44.394647 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:44.395103 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:44.396200 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:44.397499 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:44.398171 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:44.398285 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:44.399257 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:44.399263 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:44.399267 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:44.399288 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:44.399333 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:44.399353 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:44.399462 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:44.399586 (+ 124us) server_negotiation.cc:299] Negotiation successful
0928 14:55:44.399644 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 14:55:44.399999 4800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:44.402107 4790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:44.406361 4800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [term 1 LEADER]: Becoming Leader. State: Replica: 1fe6528b287a4ac1ab4f94fe65db0ad7, State: Running, Role: LEADER
I20260928 14:55:44.406870 4800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe6528b287a4ac1ab4f94fe65db0ad7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41033 } }
I20260928 14:55:44.409858 4802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fe6528b287a4ac1ab4f94fe65db0ad7. Latest consensus state: current_term: 1 leader_uuid: "1fe6528b287a4ac1ab4f94fe65db0ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe6528b287a4ac1ab4f94fe65db0ad7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41033 } } }
I20260928 14:55:44.409771 4801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fe6528b287a4ac1ab4f94fe65db0ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe6528b287a4ac1ab4f94fe65db0ad7" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 41033 } } }
I20260928 14:55:44.410193 4802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:44.410198 4801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7 [sys.catalog]: This master's current role is: LEADER
W20260928 14:55:44.415182 4814 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:55:44.415288 4814 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:55:44.415400 4815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:44.417528 4815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:44.426745 4815 catalog_manager.cc:1347] Generated new cluster ID: 1aabdf1222614b1ebb999494235c9608
I20260928 14:55:44.426852 4815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:44.461797 4815 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:44.463078 4815 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:44.471153 4815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fe6528b287a4ac1ab4f94fe65db0ad7: Generated new TSK 0
I20260928 14:55:44.471983 4815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:44.480063 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4722
W20260928 14:55:44.481230 4785 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50267
W20260928 14:55:44.481932 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4722: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:44.482054 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 4722
W20260928 14:55:44.496148 4707 connection.cc:535] client connection to 127.14.241.254:41033 recv error: Network error: failed to read from TLS socket (remote: 127.14.241.254:41033): Connection reset by peer (error 104)
2026-09-28T14:55:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1794 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:44 dist-test-slave-vk40 krb5kdc[4821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:44 dist-test-slave-vk40 krb5kdc[4821](info): set up 2 sockets
Sep 28 14:55:44 dist-test-slave-vk40 krb5kdc[4821](info): commencing operation
krb5kdc: starting...
W20260928 14:55:45.503338 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:45 dist-test-slave-vk40 krb5kdc[4821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:55:45.753021 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:34705
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:34705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:46.076462 4835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:46.076773 4835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:46.076872 4835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:46.084419 4835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:46.084553 4835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:46.084630 4835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:46.084722 4835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:46.084777 4835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:46.096050 4835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:34705
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:34705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:46.097402 4835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:46.098737 4835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:46.110021 4843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:46.110061 4840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:46.110266 4841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:46.111032 4835 server_base.cc:1034] running on GCE node
I20260928 14:55:46.111829 4835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:46.113315 4835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:46.114539 4835 hybrid_clock.cc:648] HybridClock initialized: now 1790607346114483 us; error 46 us; skew 500 ppm
I20260928 14:55:46.114899 4835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:46 dist-test-slave-vk40 krb5kdc[4821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607346, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:46.122364 4835 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:55:46.123941 4835 webserver.cc:460] Webserver started at http://127.14.241.254:39755/ using document root <none> and password file <none>
I20260928 14:55:46.124691 4835 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:46.124805 4835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:46.125108 4835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:46.128075 4835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "1a03b2ac2b75447abf2a26d0867d0a1f"
format_stamp: "Formatted at 2026-09-28 14:55:46 on dist-test-slave-vk40"
server_key: "24281e6d3b727275e9201aca69da7188"
server_key_iv: "d7fc60099f4ea8a9c2189a2f567cdf89"
server_key_version: "encryptionkey@0"
I20260928 14:55:46.128860 4835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "1a03b2ac2b75447abf2a26d0867d0a1f"
format_stamp: "Formatted at 2026-09-28 14:55:46 on dist-test-slave-vk40"
server_key: "24281e6d3b727275e9201aca69da7188"
server_key_iv: "d7fc60099f4ea8a9c2189a2f567cdf89"
server_key_version: "encryptionkey@0"
I20260928 14:55:46.133847 4835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 14:55:46.137204 4850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:46.138535 4835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:55:46.138688 4835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "1a03b2ac2b75447abf2a26d0867d0a1f"
format_stamp: "Formatted at 2026-09-28 14:55:46 on dist-test-slave-vk40"
server_key: "24281e6d3b727275e9201aca69da7188"
server_key_iv: "d7fc60099f4ea8a9c2189a2f567cdf89"
server_key_version: "encryptionkey@0"
I20260928 14:55:46.138872 4835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:46.144558 4835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:46.148706 4835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:46.148995 4835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:46.166363 4835 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:34705
I20260928 14:55:46.166399 4901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:34705 every 8 connection(s)
I20260928 14:55:46.167766 4835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:46.171435 4902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:46.176338 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 4835
I20260928 14:55:46.176522 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:46.177075 15303 external_mini_cluster.cc:1428] Setting key 0e0234471158585fc30a30e043f05ba2
I20260928 14:55:46.183163 4902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f: Bootstrap starting.
I20260928 14:55:46.185019 4905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:46.178847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33735 (local address 127.14.241.254:34705)
0928 14:55:46.179661 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:46.179675 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:46.179718 (+ 43us) server_negotiation.cc:407] Connection header received
0928 14:55:46.181705 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:46.181728 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:46.182084 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:46.182464 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:46.183437 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:46.183443 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:46.183458 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:46.183473 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:46.183533 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:46.183553 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:46.183760 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:46.183885 (+ 125us) server_negotiation.cc:299] Negotiation successful
0928 14:55:46.184059 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":375,"threads_started":1}
I20260928 14:55:46.186565 4902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:46.187829 4902 log.cc:824] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:46.188867 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:46.190630 4902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f: No bootstrap required, opened a new log
I20260928 14:55:46.195158 4905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:46.192824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53679 (local address 127.14.241.254:34705)
0928 14:55:46.192993 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:46.192999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:46.193421 (+ 422us) server_negotiation.cc:407] Connection header received
0928 14:55:46.193552 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:46.193557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:46.193617 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:46.193734 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:46.194541 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:46.194547 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:46.194552 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:46.194570 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:46.194617 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:46.194637 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:46.194751 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:46.194796 (+ 45us) server_negotiation.cc:299] Negotiation successful
0928 14:55:46.194854 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 14:55:46.198989 4902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a03b2ac2b75447abf2a26d0867d0a1f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34705 } }
I20260928 14:55:46.199366 4902 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:46.199473 4902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a03b2ac2b75447abf2a26d0867d0a1f, State: Initialized, Role: FOLLOWER
I20260928 14:55:46.200085 4902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a03b2ac2b75447abf2a26d0867d0a1f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34705 } }
I20260928 14:55:46.200341 4902 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:46.200485 4902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:46.200644 4902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:46.202838 4902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a03b2ac2b75447abf2a26d0867d0a1f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34705 } }
I20260928 14:55:46.203334 4902 leader_election.cc:302] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a03b2ac2b75447abf2a26d0867d0a1f; no voters:
I20260928 14:55:46.203886 4902 leader_election.cc:288] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:46.204037 4912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:46.204666 4912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [term 1 LEADER]: Becoming Leader. State: Replica: 1a03b2ac2b75447abf2a26d0867d0a1f, State: Running, Role: LEADER
I20260928 14:55:46.205229 4912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a03b2ac2b75447abf2a26d0867d0a1f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34705 } }
I20260928 14:55:46.205940 4902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:46.208228 4914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a03b2ac2b75447abf2a26d0867d0a1f. Latest consensus state: current_term: 1 leader_uuid: "1a03b2ac2b75447abf2a26d0867d0a1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a03b2ac2b75447abf2a26d0867d0a1f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34705 } } }
I20260928 14:55:46.208742 4914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:46.208226 4913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a03b2ac2b75447abf2a26d0867d0a1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a03b2ac2b75447abf2a26d0867d0a1f" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 34705 } } }
I20260928 14:55:46.208998 4913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:46.210875 4919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:46.216240 4919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:46.225916 4919 catalog_manager.cc:1347] Generated new cluster ID: dd08d8a190674260b2e346d1729479ce
I20260928 14:55:46.226037 4919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:46.244917 4919 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:46.245337 4919 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:46.261713 4919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a03b2ac2b75447abf2a26d0867d0a1f: Generated new TSK 0
I20260928 14:55:46.262846 4919 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:46.271143 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4835
W20260928 14:55:46.272358 4897 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33735
W20260928 14:55:46.273030 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4835: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:46.273154 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 4835
2026-09-28T14:55:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1790 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:46 dist-test-slave-vk40 krb5kdc[4934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:46 dist-test-slave-vk40 krb5kdc[4934](info): set up 2 sockets
Sep 28 14:55:46 dist-test-slave-vk40 krb5kdc[4934](info): commencing operation
krb5kdc: starting...
W20260928 14:55:47.291211 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:47 dist-test-slave-vk40 krb5kdc[4934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:55:47.537581 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:42627
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:42627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:47.857739 4947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:47.858053 4947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:47.858186 4947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:47.865473 4947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:47.865604 4947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:47.865684 4947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:47.865772 4947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:47.865828 4947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:47.875789 4947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:42627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:42627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:47.877099 4947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:47.878448 4947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:47.890239 4952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:47.890251 4955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:47.890194 4953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:47.890973 4947 server_base.cc:1034] running on GCE node
I20260928 14:55:47.891728 4947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:47.893141 4947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:47.894346 4947 hybrid_clock.cc:648] HybridClock initialized: now 1790607347894303 us; error 40 us; skew 500 ppm
I20260928 14:55:47.894680 4947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:47 dist-test-slave-vk40 krb5kdc[4934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607347, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:47.897668 4947 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:55:47.899065 4947 webserver.cc:460] Webserver started at http://127.14.241.254:41857/ using document root <none> and password file <none>
I20260928 14:55:47.899767 4947 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:47.899879 4947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:47.900130 4947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:47.903070 4947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "232667c14591429da183346354a08fcf"
format_stamp: "Formatted at 2026-09-28 14:55:47 on dist-test-slave-vk40"
server_key: "ff5d2df50902838b203a588d719349d9"
server_key_iv: "9c1019a91ef8044b671d5e1dcb082eaf"
server_key_version: "encryptionkey@0"
I20260928 14:55:47.903810 4947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "232667c14591429da183346354a08fcf"
format_stamp: "Formatted at 2026-09-28 14:55:47 on dist-test-slave-vk40"
server_key: "ff5d2df50902838b203a588d719349d9"
server_key_iv: "9c1019a91ef8044b671d5e1dcb082eaf"
server_key_version: "encryptionkey@0"
I20260928 14:55:47.908838 4947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260928 14:55:47.912349 4962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:47.913686 4947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 14:55:47.913826 4947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "232667c14591429da183346354a08fcf"
format_stamp: "Formatted at 2026-09-28 14:55:47 on dist-test-slave-vk40"
server_key: "ff5d2df50902838b203a588d719349d9"
server_key_iv: "9c1019a91ef8044b671d5e1dcb082eaf"
server_key_version: "encryptionkey@0"
I20260928 14:55:47.914000 4947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:47.933611 4947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:47.937403 4947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:47.937645 4947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:47.954615 4947 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:42627
I20260928 14:55:47.954633 5013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:42627 every 8 connection(s)
I20260928 14:55:47.956048 4947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:47.959933 5014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:47.960929 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 4947
I20260928 14:55:47.961077 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:47.961519 15303 external_mini_cluster.cc:1428] Setting key d57707df2328a9a10a1072a75bb963f3
I20260928 14:55:47.974120 5014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf: Bootstrap starting.
Sep 28 14:55:47 dist-test-slave-vk40 krb5kdc[4934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:47.977310 5014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:47.978650 5014 log.cc:824] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:47.981936 5014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf: No bootstrap required, opened a new log
I20260928 14:55:47.985154 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:47.963346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36529 (local address 127.14.241.254:42627)
0928 14:55:47.964167 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:47.964183 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:47.964238 (+ 55us) server_negotiation.cc:407] Connection header received
0928 14:55:47.966251 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:47.966280 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:47.966865 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:47.967340 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:47.968657 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:47.970645 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:47.971937 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:47.973010 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:47.973194 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:47.977011 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:47.977021 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:47.977041 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:47.977087 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:47.980630 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:47.981294 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:47.981301 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:47.981310 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:47.981534 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:47.981873 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:47.981877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:47.981880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:47.982612 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:47.982925 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:47.983274 (+ 349us) server_negotiation.cc:299] Negotiation successful
0928 14:55:47.983731 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":327,"threads_started":1}
I20260928 14:55:47.989214 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:47.991171 5014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232667c14591429da183346354a08fcf" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42627 } }
I20260928 14:55:47.991532 5014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:47.991627 5014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 232667c14591429da183346354a08fcf, State: Initialized, Role: FOLLOWER
I20260928 14:55:47.992300 5014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232667c14591429da183346354a08fcf" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42627 } }
I20260928 14:55:47.992564 5014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:47.992691 5014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:47.992846 5014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:47.996037 5014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232667c14591429da183346354a08fcf" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42627 } }
I20260928 14:55:47.996587 5014 leader_election.cc:302] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 232667c14591429da183346354a08fcf; no voters:
I20260928 14:55:47.997207 5014 leader_election.cc:288] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:47.997308 5024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:47.998428 5024 raft_consensus.cc:686] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [term 1 LEADER]: Becoming Leader. State: Replica: 232667c14591429da183346354a08fcf, State: Running, Role: LEADER
I20260928 14:55:47.999083 5024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232667c14591429da183346354a08fcf" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42627 } }
I20260928 14:55:47.999486 5014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:48.002276 5025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "232667c14591429da183346354a08fcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232667c14591429da183346354a08fcf" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42627 } } }
I20260928 14:55:48.002691 5025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:48.003829 5026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 232667c14591429da183346354a08fcf. Latest consensus state: current_term: 1 leader_uuid: "232667c14591429da183346354a08fcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "232667c14591429da183346354a08fcf" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 42627 } } }
I20260928 14:55:48.004132 5026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:48.006307 5030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:48.010982 5030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:48.012390 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:47.993253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59491 (local address 127.14.241.254:42627)
0928 14:55:47.993454 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:47.993465 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:47.993861 (+ 396us) server_negotiation.cc:407] Connection header received
0928 14:55:47.994019 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:47.994022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:47.994086 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:47.994164 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:47.995934 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:47.996678 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:47.998105 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:47.998793 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:47.998899 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:48.001402 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:48.001409 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:48.001413 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:48.001455 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:48.005047 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:48.005878 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:48.005884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:48.005888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:48.006042 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:48.006473 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:48.006478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:48.006482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:48.006911 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:48.007163 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:48.007737 (+ 574us) server_negotiation.cc:299] Negotiation successful
0928 14:55:48.007953 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 14:55:48.020442 5030 catalog_manager.cc:1347] Generated new cluster ID: 683de923863b426a9340c5f775e6bc03
I20260928 14:55:48.020793 5030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:48.029676 5030 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:48.031584 5030 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:48.049405 5030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 232667c14591429da183346354a08fcf: Generated new TSK 0
I20260928 14:55:48.050491 5030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:48.060922 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 4947
I20260928 14:55:48.062288 5009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:48.155052 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 4947
2026-09-28T14:55:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1882 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:48 dist-test-slave-vk40 krb5kdc[5046](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:48 dist-test-slave-vk40 krb5kdc[5046](info): set up 2 sockets
Sep 28 14:55:48 dist-test-slave-vk40 krb5kdc[5046](info): commencing operation
krb5kdc: starting...
W20260928 14:55:49.179323 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:49 dist-test-slave-vk40 krb5kdc[5046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:55:49.431317 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:43411
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:43411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:49.755955 5059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:49.756239 5059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:49.756389 5059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:49.763541 5059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:49.763667 5059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:49.763756 5059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:49.763836 5059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:49.763896 5059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:49.773871 5059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:43411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:43411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:49.775164 5059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:49.776485 5059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:49.788031 5064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:49.788059 5065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:49.787988 5067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:49.788847 5059 server_base.cc:1034] running on GCE node
I20260928 14:55:49.789597 5059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:49.790927 5059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:49.792138 5059 hybrid_clock.cc:648] HybridClock initialized: now 1790607349792083 us; error 41 us; skew 500 ppm
I20260928 14:55:49.792496 5059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:49 dist-test-slave-vk40 krb5kdc[5046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607349, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:49.795425 5059 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:55:49.796859 5059 webserver.cc:460] Webserver started at http://127.14.241.254:38107/ using document root <none> and password file <none>
I20260928 14:55:49.797505 5059 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:49.797621 5059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:49.797902 5059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:49.800928 5059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "4f47ce4e9b1a4ff4b5a1b1acb44dfeea"
format_stamp: "Formatted at 2026-09-28 14:55:49 on dist-test-slave-vk40"
server_key: "bce13d40a2236000fbebacab46eb661c"
server_key_iv: "34417714d3cb8e1a9090f892b45d9099"
server_key_version: "encryptionkey@0"
I20260928 14:55:49.801692 5059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "4f47ce4e9b1a4ff4b5a1b1acb44dfeea"
format_stamp: "Formatted at 2026-09-28 14:55:49 on dist-test-slave-vk40"
server_key: "bce13d40a2236000fbebacab46eb661c"
server_key_iv: "34417714d3cb8e1a9090f892b45d9099"
server_key_version: "encryptionkey@0"
I20260928 14:55:49.806944 5059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:49.810604 5075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:49.811966 5059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:55:49.812137 5059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "4f47ce4e9b1a4ff4b5a1b1acb44dfeea"
format_stamp: "Formatted at 2026-09-28 14:55:49 on dist-test-slave-vk40"
server_key: "bce13d40a2236000fbebacab46eb661c"
server_key_iv: "34417714d3cb8e1a9090f892b45d9099"
server_key_version: "encryptionkey@0"
I20260928 14:55:49.812345 5059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:49.825704 5059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:49.829552 5059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:49.829823 5059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:49.847147 5059 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:43411
I20260928 14:55:49.847167 5127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:43411 every 8 connection(s)
I20260928 14:55:49.848589 5059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:49.852210 5128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:49.855849 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 5059
I20260928 14:55:49.856034 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:49.856595 15303 external_mini_cluster.cc:1428] Setting key 96cb176a88094a2ad1c186816cc14c36
I20260928 14:55:49.864259 5128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea: Bootstrap starting.
I20260928 14:55:49.867390 5128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:49.868494 5128 log.cc:824] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:49.869316 5131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:49.858421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39157 (local address 127.14.241.254:43411)
0928 14:55:49.859206 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:49.859220 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:49.859260 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:55:49.860973 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:49.860997 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:49.861441 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:49.861813 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:49.862367 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:49.863687 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:49.865054 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:49.865730 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:49.865887 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:49.867031 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:49.867038 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:49.867052 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:49.867591 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:49.867672 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:49.867706 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:49.867824 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:49.868013 (+ 189us) server_negotiation.cc:299] Negotiation successful
0928 14:55:49.868261 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":347,"threads_started":1}
I20260928 14:55:49.871133 5128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea: No bootstrap required, opened a new log
I20260928 14:55:49.872635 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:49.879320 5128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f47ce4e9b1a4ff4b5a1b1acb44dfeea" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43411 } }
I20260928 14:55:49.879704 5128 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:49.879799 5128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f47ce4e9b1a4ff4b5a1b1acb44dfeea, State: Initialized, Role: FOLLOWER
I20260928 14:55:49.880434 5128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f47ce4e9b1a4ff4b5a1b1acb44dfeea" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43411 } }
I20260928 14:55:49.880724 5128 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:49.880877 5128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:49.881062 5128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:49.882910 5131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:49.876542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44265 (local address 127.14.241.254:43411)
0928 14:55:49.876717 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:49.876723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:49.877248 (+ 525us) server_negotiation.cc:407] Connection header received
0928 14:55:49.877383 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:49.877389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:49.877471 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:49.877586 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:49.878000 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:49.879035 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:49.880347 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:49.881002 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:49.881225 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:49.882207 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:49.882213 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:49.882217 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:49.882239 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:49.882299 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:49.882321 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:49.882413 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:49.882491 (+ 78us) server_negotiation.cc:299] Negotiation successful
0928 14:55:49.882588 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 14:55:49.883888 5128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f47ce4e9b1a4ff4b5a1b1acb44dfeea" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43411 } }
I20260928 14:55:49.884433 5128 leader_election.cc:302] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f47ce4e9b1a4ff4b5a1b1acb44dfeea; no voters:
I20260928 14:55:49.884934 5128 leader_election.cc:288] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:49.885061 5139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:49.885664 5139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [term 1 LEADER]: Becoming Leader. State: Replica: 4f47ce4e9b1a4ff4b5a1b1acb44dfeea, State: Running, Role: LEADER
I20260928 14:55:49.886186 5139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f47ce4e9b1a4ff4b5a1b1acb44dfeea" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43411 } }
I20260928 14:55:49.886919 5128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:49.888906 5140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f47ce4e9b1a4ff4b5a1b1acb44dfeea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f47ce4e9b1a4ff4b5a1b1acb44dfeea" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43411 } } }
I20260928 14:55:49.889302 5140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:49.891570 5145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:49.891391 5141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f47ce4e9b1a4ff4b5a1b1acb44dfeea. Latest consensus state: current_term: 1 leader_uuid: "4f47ce4e9b1a4ff4b5a1b1acb44dfeea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f47ce4e9b1a4ff4b5a1b1acb44dfeea" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43411 } } }
I20260928 14:55:49.891698 5141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:49.896509 5145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:49.906567 5145 catalog_manager.cc:1347] Generated new cluster ID: 6d5dd501d83a41759e002c639f29c2b3
I20260928 14:55:49.906705 5145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:49.926476 5145 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:49.928368 5145 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:49.936146 5145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f47ce4e9b1a4ff4b5a1b1acb44dfeea: Generated new TSK 0
I20260928 14:55:49.937194 5145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:49.947371 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 5059
W20260928 14:55:49.948544 5123 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39157
W20260928 14:55:49.949169 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 5059: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:49.949328 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 5059
2026-09-28T14:55:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1793 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:50 dist-test-slave-vk40 krb5kdc[5160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:50 dist-test-slave-vk40 krb5kdc[5160](info): set up 2 sockets
Sep 28 14:55:50 dist-test-slave-vk40 krb5kdc[5160](info): commencing operation
krb5kdc: starting...
W20260928 14:55:50.967315 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.003s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:51 dist-test-slave-vk40 krb5kdc[5160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:55:51.227385 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:40913
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:40633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:40913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:51.555574 5173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:51.555886 5173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:51.556025 5173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:51.563297 5173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:51.563423 5173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:51.563506 5173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:51.563592 5173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:51.563655 5173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:51.573613 5173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:40633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:40913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:40913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:51.574934 5173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:51.576225 5173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:51.587594 5178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:51.587558 5181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:51.587628 5179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:51.588544 5173 server_base.cc:1034] running on GCE node
I20260928 14:55:51.589330 5173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:51.590822 5173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:51.592036 5173 hybrid_clock.cc:648] HybridClock initialized: now 1790607351591986 us; error 45 us; skew 500 ppm
I20260928 14:55:51.592417 5173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:51 dist-test-slave-vk40 krb5kdc[5160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607351, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:51.602545 5173 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:55:51.604058 5173 webserver.cc:460] Webserver started at http://127.14.241.254:34671/ using document root <none> and password file <none>
I20260928 14:55:51.604779 5173 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:51.604884 5173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:51.605139 5173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:51.608151 5173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "a02819d1180e433d88b7f74db2ce6f03"
format_stamp: "Formatted at 2026-09-28 14:55:51 on dist-test-slave-vk40"
server_key: "c38129720b73b6f091c9fd0cb7949db0"
server_key_iv: "b3c8adeef815e66049e8b20df06b04ac"
server_key_version: "encryptionkey@0"
I20260928 14:55:51.608945 5173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "a02819d1180e433d88b7f74db2ce6f03"
format_stamp: "Formatted at 2026-09-28 14:55:51 on dist-test-slave-vk40"
server_key: "c38129720b73b6f091c9fd0cb7949db0"
server_key_iv: "b3c8adeef815e66049e8b20df06b04ac"
server_key_version: "encryptionkey@0"
I20260928 14:55:51.614563 5173 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:51.618156 5188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:51.619549 5173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:55:51.619691 5173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "a02819d1180e433d88b7f74db2ce6f03"
format_stamp: "Formatted at 2026-09-28 14:55:51 on dist-test-slave-vk40"
server_key: "c38129720b73b6f091c9fd0cb7949db0"
server_key_iv: "b3c8adeef815e66049e8b20df06b04ac"
server_key_version: "encryptionkey@0"
I20260928 14:55:51.619872 5173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:51.625556 5173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:51.629822 5173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:51.630084 5173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:51.647725 5173 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:40913
I20260928 14:55:51.647802 5240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:40913 every 8 connection(s)
I20260928 14:55:51.649189 5173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:51.651214 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 5173
I20260928 14:55:51.651420 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:51.651952 15303 external_mini_cluster.cc:1428] Setting key e9ab035821599cdabbe3d7269dbeb79a
I20260928 14:55:51.653791 5241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:51.659627 5244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:51.653730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33241 (local address 127.14.241.254:40913)
0928 14:55:51.654515 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:51.654528 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:51.654569 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:55:51.656186 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:51.656211 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:51.656630 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:51.656997 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:51.658030 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:51.658037 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:51.658051 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:51.658067 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:51.658143 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:51.658162 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:51.658345 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:51.658453 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 14:55:51.658643 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":354,"threads_started":1}
I20260928 14:55:51.663192 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:51.666435 5241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03: Bootstrap starting.
I20260928 14:55:51.670534 5241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:51.670590 5244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:51.668342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59947 (local address 127.14.241.254:40913)
0928 14:55:51.668529 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:51.668536 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:51.668832 (+ 296us) server_negotiation.cc:407] Connection header received
0928 14:55:51.669003 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:51.669006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:51.669055 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:51.669131 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:51.670000 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:51.670004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:51.670007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:55:51.670021 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:51.670058 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:55:51.670072 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:51.670155 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:51.670270 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 14:55:51.670316 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 14:55:51.671653 5241 log.cc:824] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:51.674371 5241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03: No bootstrap required, opened a new log
I20260928 14:55:51.681860 5241 raft_consensus.cc:348] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02819d1180e433d88b7f74db2ce6f03" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40913 } }
I20260928 14:55:51.682274 5241 raft_consensus.cc:374] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:55:51.682374 5241 raft_consensus.cc:729] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02819d1180e433d88b7f74db2ce6f03, State: Initialized, Role: FOLLOWER
I20260928 14:55:51.682917 5241 consensus_queue.cc:260] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02819d1180e433d88b7f74db2ce6f03" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40913 } }
I20260928 14:55:51.683125 5241 raft_consensus.cc:388] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:55:51.683243 5241 raft_consensus.cc:482] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:55:51.683394 5241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:51.685586 5241 raft_consensus.cc:504] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02819d1180e433d88b7f74db2ce6f03" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40913 } }
I20260928 14:55:51.686055 5241 leader_election.cc:302] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a02819d1180e433d88b7f74db2ce6f03; no voters:
I20260928 14:55:51.686592 5241 leader_election.cc:288] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:55:51.686764 5251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:51.687374 5251 raft_consensus.cc:686] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [term 1 LEADER]: Becoming Leader. State: Replica: a02819d1180e433d88b7f74db2ce6f03, State: Running, Role: LEADER
I20260928 14:55:51.687937 5251 consensus_queue.cc:237] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02819d1180e433d88b7f74db2ce6f03" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40913 } }
I20260928 14:55:51.688548 5241 sys_catalog.cc:564] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:51.690757 5252 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a02819d1180e433d88b7f74db2ce6f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02819d1180e433d88b7f74db2ce6f03" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40913 } } }
I20260928 14:55:51.691200 5252 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:51.691340 5253 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a02819d1180e433d88b7f74db2ce6f03. Latest consensus state: current_term: 1 leader_uuid: "a02819d1180e433d88b7f74db2ce6f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02819d1180e433d88b7f74db2ce6f03" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 40913 } } }
I20260928 14:55:51.691756 5253 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:51.692510 5260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:51.702756 5260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:51.712366 5260 catalog_manager.cc:1347] Generated new cluster ID: d4250732cea140529708f42273a5202b
I20260928 14:55:51.712481 5260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:51.727418 5260 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:51.727823 5260 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:51.737103 5260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a02819d1180e433d88b7f74db2ce6f03: Generated new TSK 0
I20260928 14:55:51.737920 5260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:55:51.746137 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 5173
W20260928 14:55:51.747440 5236 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33241
W20260928 14:55:51.748172 15303 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 5173: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260928 14:55:51.748332 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 5173
2026-09-28T14:55:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1798 ms)
[----------] 18 tests from AuthTokenIssuingTest (33127 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:51 dist-test-slave-vk40 krb5kdc[5272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:51 dist-test-slave-vk40 krb5kdc[5272](info): set up 2 sockets
Sep 28 14:55:51 dist-test-slave-vk40 krb5kdc[5272](info): commencing operation
krb5kdc: starting...
W20260928 14:55:52.767549 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:52 dist-test-slave-vk40 krb5kdc[5272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:55:53.024783 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:43445
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:43445,127.14.241.253:37305,127.14.241.252:42013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:53.356341 5285 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:55:53.356720 5285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:53.356817 5285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:53.356927 5285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:53.364476 5285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:53.364588 5285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:53.364682 5285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:53.364748 5285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:53.364831 5285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:53.375134 5285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:43445,127.14.241.253:37305,127.14.241.252:42013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:43445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:53.376509 5285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:53.377784 5285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:53.389825 5290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:53.389906 5291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:53.389863 5293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:53.390477 5285 server_base.cc:1034] running on GCE node
I20260928 14:55:53.391261 5285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:53.392727 5285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:53.393957 5285 hybrid_clock.cc:648] HybridClock initialized: now 1790607353393897 us; error 27 us; skew 500 ppm
I20260928 14:55:53.394287 5285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:53 dist-test-slave-vk40 krb5kdc[5272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607353, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:53.397430 5285 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:55:53.399010 5285 webserver.cc:460] Webserver started at http://127.14.241.254:36431/ using document root <none> and password file <none>
I20260928 14:55:53.399676 5285 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:53.399761 5285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:53.400027 5285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:53.403005 5285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "f60590b979bf4055b858e74d0fa65598"
format_stamp: "Formatted at 2026-09-28 14:55:53 on dist-test-slave-vk40"
server_key: "cde5108c2653aadca04cba09ecb3124f"
server_key_iv: "1510513b35dc66f9ad7cdb67b0cc3b4c"
server_key_version: "encryptionkey@0"
I20260928 14:55:53.403755 5285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "f60590b979bf4055b858e74d0fa65598"
format_stamp: "Formatted at 2026-09-28 14:55:53 on dist-test-slave-vk40"
server_key: "cde5108c2653aadca04cba09ecb3124f"
server_key_iv: "1510513b35dc66f9ad7cdb67b0cc3b4c"
server_key_version: "encryptionkey@0"
I20260928 14:55:53.409061 5285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260928 14:55:53.412681 5300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:53.414136 5285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260928 14:55:53.414311 5285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "f60590b979bf4055b858e74d0fa65598"
format_stamp: "Formatted at 2026-09-28 14:55:53 on dist-test-slave-vk40"
server_key: "cde5108c2653aadca04cba09ecb3124f"
server_key_iv: "1510513b35dc66f9ad7cdb67b0cc3b4c"
server_key_version: "encryptionkey@0"
I20260928 14:55:53.414538 5285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:53.434505 5285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:53.438736 5285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:53.439002 5285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:53.456017 5285 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:43445
I20260928 14:55:53.456035 5351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:43445 every 8 connection(s)
I20260928 14:55:53.457368 5285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:53.459581 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 5285
I20260928 14:55:53.459806 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:53.460479 15303 external_mini_cluster.cc:1428] Setting key e7cf3aa60c7980f68a669023c6993865
I20260928 14:55:53.461393 5352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:53.468338 5352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:55:53 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607353, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.241.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.253@KRBTEST.COM" created.
I20260928 14:55:53.491400 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.472104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53750 (local address 127.14.241.254:43445)
0928 14:55:53.475495 (+ 3391us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:53.475501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:53.475529 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:55:53.475658 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:53.475663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:53.475876 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:53.476054 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:53.477374 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:53.478554 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:53.479911 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:53.480579 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:53.480749 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:53.483939 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:53.483948 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:53.483961 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:53.483994 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:53.487055 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:53.487728 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:53.487736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:53.487740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:53.487889 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:53.488245 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:53.488249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:53.488251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:53.488801 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:53.488997 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:53.489533 (+ 536us) server_negotiation.cc:299] Negotiation successful
0928 14:55:53.489900 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":3234,"spinlock_wait_cycles":22400,"thread_start_us":3055,"threads_started":1}
I20260928 14:55:53.491433 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.472104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43445 (local address 127.0.0.1:53750)
0928 14:55:53.472957 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.473001 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:53.474877 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:53.476095 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:53.476103 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:53.476841 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:53.477173 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:53.477191 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:53.478692 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:53.478696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:53.479781 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:53.479793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:53.480808 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:53.480815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:53.480986 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:53.481570 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:53.481593 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:53.483703 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:53.487203 (+ 3500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:53.487214 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:53.487225 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:53.487600 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:53.487995 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:53.487998 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:53.488001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:53.488140 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:53.488937 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:53.488945 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:53.489154 (+ 209us) client_negotiation.cc:769] Sending connection context
0928 14:55:53.489391 (+ 237us) client_negotiation.cc:240] Negotiation successful
0928 14:55:53.490032 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":135,"thread_start_us":366,"threads_started":1}
I20260928 14:55:53.494695 5352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:55:53.495886 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.495466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51072)
0928 14:55:53.495633 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.495770 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260928 14:55:53.496146 5301 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
I20260928 14:55:53.497687 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.497334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51074)
0928 14:55:53.497475 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.497542 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 14:55:53.497864 5352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:37305: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab.
Entry for principal kudu/127.14.241.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab.
I20260928 14:55:53.519160 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.253:37305
--webserver_interface=127.14.241.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab
--principal=kudu/127.14.241.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:43445,127.14.241.253:37305,127.14.241.252:42013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:55:53.547230 5352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } attempt: 1
I20260928 14:55:53.548450 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.547947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51090)
0928 14:55:53.548155 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.548273 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260928 14:55:53.549451 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.549132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51094)
0928 14:55:53.549247 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.549318 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 14:55:53.549641 5352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:37305: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
I20260928 14:55:53.617916 5352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } attempt: 2
I20260928 14:55:53.619131 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.618612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51096)
0928 14:55:53.618857 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.618983 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260928 14:55:53.620061 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.619709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51098)
0928 14:55:53.619858 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.619929 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 14:55:53.620242 5352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:37305: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
I20260928 14:55:53.711534 5352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } attempt: 3
I20260928 14:55:53.712790 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.712361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51108)
0928 14:55:53.712554 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.712647 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 14:55:53.713759 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.713405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51114)
0928 14:55:53.713547 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.713618 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260928 14:55:53.713933 5352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:37305: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
W20260928 14:55:53.846565 5359 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:55:53.846946 5359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:53.847043 5359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:53.847173 5359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:53.854588 5359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:53.854693 5359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:53.854775 5359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:53.854837 5359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:53.854921 5359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:53.857147 5352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } attempt: 4
I20260928 14:55:53.858342 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.857806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51122)
0928 14:55:53.858113 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.858204 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260928 14:55:53.859344 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.858960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51136)
0928 14:55:53.859114 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.859207 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 14:55:53.859519 5352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:37305: Network error: Client connection negotiation failed: client connection to 127.14.241.253:37305: connect: Connection refused (error 111)
I20260928 14:55:53.865456 5359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:43445,127.14.241.253:37305,127.14.241.252:42013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab
--principal=kudu/127.14.241.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.253:37305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.241.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:53.866757 5359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:53.868046 5359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:53.879483 5364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:53.879519 5365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:53.879554 5367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:53.880391 5359 server_base.cc:1034] running on GCE node
I20260928 14:55:53.881129 5359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:53.882501 5359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:53.883715 5359 hybrid_clock.cc:648] HybridClock initialized: now 1790607353883654 us; error 33 us; skew 500 ppm
I20260928 14:55:53.884056 5359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:53 dist-test-slave-vk40 krb5kdc[5272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607353, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:53.887333 5359 init.cc:377] Logged in from keytab as kudu/127.14.241.253@KRBTEST.COM (short username kudu)
I20260928 14:55:53.888866 5359 webserver.cc:460] Webserver started at http://127.14.241.253:46005/ using document root <none> and password file <none>
I20260928 14:55:53.889562 5359 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:53.889678 5359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:53.889981 5359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:53.892990 5359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/data/instance:
uuid: "0092a1ccec03481095e99083594a7950"
format_stamp: "Formatted at 2026-09-28 14:55:53 on dist-test-slave-vk40"
server_key: "5ccf6af01b71edd5de005c8f82baafbb"
server_key_iv: "ed3faa518de6a4c5735b62613cce25cf"
server_key_version: "encryptionkey@0"
I20260928 14:55:53.893774 5359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/wal/instance:
uuid: "0092a1ccec03481095e99083594a7950"
format_stamp: "Formatted at 2026-09-28 14:55:53 on dist-test-slave-vk40"
server_key: "5ccf6af01b71edd5de005c8f82baafbb"
server_key_iv: "ed3faa518de6a4c5735b62613cce25cf"
server_key_version: "encryptionkey@0"
I20260928 14:55:53.898892 5359 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:53.902382 5374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:53.903685 5359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 14:55:53.903851 5359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/wal
uuid: "0092a1ccec03481095e99083594a7950"
format_stamp: "Formatted at 2026-09-28 14:55:53 on dist-test-slave-vk40"
server_key: "5ccf6af01b71edd5de005c8f82baafbb"
server_key_iv: "ed3faa518de6a4c5735b62613cce25cf"
server_key_version: "encryptionkey@0"
I20260928 14:55:53.904050 5359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:53.940660 5359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:53.945073 5359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:53.945340 5359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:53.962764 5359 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.253:37305
I20260928 14:55:53.962786 5426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.253:37305 every 8 connection(s)
I20260928 14:55:53.964073 5359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
I20260928 14:55:53.964351 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 5359
I20260928 14:55:53.964506 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-1/wal/instance
I20260928 14:55:53.964896 15303 external_mini_cluster.cc:1428] Setting key 76e540da315bc7fff42a76a5a8908591
I20260928 14:55:53.967928 5427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:53.974694 5427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.241.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.252@KRBTEST.COM" created.
Sep 28 14:55:53 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607353, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:53.996793 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.977472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.14.241.254:43445)
0928 14:55:53.977641 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:53.977646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:53.979067 (+ 1421us) server_negotiation.cc:407] Connection header received
0928 14:55:53.981112 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:53.981119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:53.981202 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:53.981304 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:53.982901 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:53.983581 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:53.985335 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:53.985745 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:53.985861 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:53.990582 (+ 4721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:53.990588 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:53.990591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:53.990629 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:53.993273 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:53.994229 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:53.994232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:53.994234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:53.994361 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:53.994878 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:53.994882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:53.994885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:53.995213 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:53.995337 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:53.996367 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0928 14:55:53.996557 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:55:53.998057 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:53.978096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43445 (local address 127.0.0.1:53766)
0928 14:55:53.978871 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:53.978918 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:53.980810 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:53.981552 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:53.981562 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:53.982112 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:53.982704 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:53.982721 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:53.983740 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:53.983744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:53.985189 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:53.985201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:53.985966 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:53.985974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:53.986200 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:53.986840 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:53.986864 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:53.990292 (+ 3428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:53.993437 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:53.993447 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:53.993466 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:53.994060 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:53.994458 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:53.994464 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:53.994467 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:53.994709 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:53.995340 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:53.995349 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:53.995903 (+ 554us) client_negotiation.cc:769] Sending connection context
0928 14:55:53.996211 (+ 308us) client_negotiation.cc:240] Negotiation successful
0928 14:55:53.996765 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":315,"threads_started":1}
I20260928 14:55:53.999933 5427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:55:54 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607353, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
I20260928 14:55:54.014976 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.000686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51142)
0928 14:55:54.000840 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.000858 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:54.000948 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:54.001742 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:54.001747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:54.002153 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:54.002294 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.002303 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.003635 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.003641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.004523 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.004534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.005307 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.005313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.005483 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.006029 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:54.006046 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:54.008347 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:54.012515 (+ 4168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.012519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.012523 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.012889 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.013347 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.013351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.013353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.013486 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.014168 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:54.014171 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:54.014307 (+ 136us) client_negotiation.cc:769] Sending connection context
0928 14:55:54.014415 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 14:55:54.014667 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 14:55:54.015070 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.000782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51142 (local address 127.14.241.253:37305)
0928 14:55:54.001342 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.001348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.001371 (+ 23us) server_negotiation.cc:407] Connection header received
0928 14:55:54.001418 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.001424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.001603 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.001730 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.002451 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.003511 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.004640 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.005169 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.005276 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.008581 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.008595 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.008612 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.008654 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.012362 (+ 3708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.013040 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.013050 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.013053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.013219 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.013588 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.013594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.013597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.014055 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.014223 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.014524 (+ 301us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.014760 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":417,"thread_start_us":284,"threads_started":1}
I20260928 14:55:54.017516 5427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:55:54.018677 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.018205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58186)
0928 14:55:54.018411 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.018507 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260928 14:55:54.018932 5378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
I20260928 14:55:54.020589 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.020262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58200)
0928 14:55:54.020421 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.020491 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260928 14:55:54.020804 5427 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:42013: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab.
Entry for principal kudu/127.14.241.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab.
I20260928 14:55:54.023514 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.252:42013
--webserver_interface=127.14.241.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab
--principal=kudu/127.14.241.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:43445,127.14.241.253:37305,127.14.241.252:42013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:55:54.070161 5427 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } attempt: 1
I20260928 14:55:54.071470 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.070928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58206)
0928 14:55:54.071119 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.071279 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 14:55:54.072623 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.072219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58212)
0928 14:55:54.072416 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.072505 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260928 14:55:54.072861 5427 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:42013: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
I20260928 14:55:54.141063 5427 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } attempt: 2
I20260928 14:55:54.142254 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.141777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58216)
0928 14:55:54.141986 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.142098 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260928 14:55:54.143190 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.142900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58220)
0928 14:55:54.143010 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.143079 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 14:55:54.143469 5427 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:42013: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
I20260928 14:55:54.158749 5352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } attempt: 5
Sep 28 14:55:54 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607353, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
I20260928 14:55:54.172019 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.159471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51146)
0928 14:55:54.159664 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.159684 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:54.159842 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:54.160196 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:54.160201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:54.160604 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:54.160803 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.160812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.161821 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.161825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.162699 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.162710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.163410 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.163416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.163630 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.164216 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:54.164234 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:54.166516 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:54.169598 (+ 3082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.169604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.169609 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.170008 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.170396 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.170402 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.170406 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.170544 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.171055 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:54.171060 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:54.171213 (+ 153us) client_negotiation.cc:769] Sending connection context
0928 14:55:54.171338 (+ 125us) client_negotiation.cc:240] Negotiation successful
0928 14:55:54.171654 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260928 14:55:54.172019 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.159604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51146 (local address 127.14.241.253:37305)
0928 14:55:54.159818 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.159824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.159848 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:55:54.159962 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.159967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.160051 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.160134 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.160959 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.161676 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.162840 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.163256 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.163383 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.166742 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.166748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.166751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.166820 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.169447 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.170129 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.170133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.170135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.170282 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.170648 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.170652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.170654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.170940 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.171073 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.171491 (+ 418us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.171705 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260928 14:55:54.173373 5352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:55:54.174417 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.174030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58236)
0928 14:55:54.174192 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.174277 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 14:55:54.175273 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.174946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58244)
0928 14:55:54.175091 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.175159 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 14:55:54.175457 5352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:42013: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
I20260928 14:55:54.226675 5352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } attempt: 1
I20260928 14:55:54.227844 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.227375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58254)
0928 14:55:54.227580 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.227675 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260928 14:55:54.228804 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.228479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58264)
0928 14:55:54.228619 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.228691 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 14:55:54.229004 5352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:42013: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
I20260928 14:55:54.234651 5427 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } attempt: 3
I20260928 14:55:54.235787 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.235285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58280)
0928 14:55:54.235495 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.235598 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260928 14:55:54.236891 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.236453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58294)
0928 14:55:54.236680 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.236774 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260928 14:55:54.237085 5427 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:42013: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
I20260928 14:55:54.297209 5352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } attempt: 2
I20260928 14:55:54.298290 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.297840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58308)
0928 14:55:54.298032 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.298126 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260928 14:55:54.299198 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.298877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58316)
0928 14:55:54.299014 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.299084 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 14:55:54.299387 5352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:42013: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
W20260928 14:55:54.355414 5434 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:55:54.355782 5434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:54.355876 5434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:54.355985 5434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:54.363394 5434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:54.363512 5434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:54.363592 5434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:54.363658 5434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:54.363737 5434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:54.374147 5434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:43445,127.14.241.253:37305,127.14.241.252:42013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab
--principal=kudu/127.14.241.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.252:42013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.241.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:54.375490 5434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:54.376894 5434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:55:54.380335 5427 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } attempt: 4
I20260928 14:55:54.381546 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.380950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58320)
0928 14:55:54.381194 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.381379 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260928 14:55:54.382840 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.382319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58332)
0928 14:55:54.382576 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.382677 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260928 14:55:54.383049 5427 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:42013: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
W20260928 14:55:54.388695 5439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:54.388741 5440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:54.388660 5442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:54.389462 5434 server_base.cc:1034] running on GCE node
I20260928 14:55:54.390242 5434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:54.391599 5434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:54.392807 5434 hybrid_clock.cc:648] HybridClock initialized: now 1790607354392735 us; error 43 us; skew 500 ppm
I20260928 14:55:54.393129 5434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:54 dist-test-slave-vk40 krb5kdc[5272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607354, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:54.396145 5434 init.cc:377] Logged in from keytab as kudu/127.14.241.252@KRBTEST.COM (short username kudu)
I20260928 14:55:54.397605 5434 webserver.cc:460] Webserver started at http://127.14.241.252:33555/ using document root <none> and password file <none>
I20260928 14:55:54.398277 5434 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:54.398362 5434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:54.398624 5434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:54.401666 5434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/data/instance:
uuid: "7419c23b12db417f85eeb47c233755ca"
format_stamp: "Formatted at 2026-09-28 14:55:54 on dist-test-slave-vk40"
server_key: "6027ec08912c3816cb72b95f6bf4ebfc"
server_key_iv: "bbac2fd2f622d2cdab9a2a3ea8eb83a0"
server_key_version: "encryptionkey@0"
I20260928 14:55:54.402407 5434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/wal/instance:
uuid: "7419c23b12db417f85eeb47c233755ca"
format_stamp: "Formatted at 2026-09-28 14:55:54 on dist-test-slave-vk40"
server_key: "6027ec08912c3816cb72b95f6bf4ebfc"
server_key_iv: "bbac2fd2f622d2cdab9a2a3ea8eb83a0"
server_key_version: "encryptionkey@0"
I20260928 14:55:54.405575 5352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } attempt: 3
I20260928 14:55:54.406786 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.406214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58340)
0928 14:55:54.406458 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.406559 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260928 14:55:54.407847 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.407478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58350)
0928 14:55:54.407645 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.407737 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260928 14:55:54.407863 5434 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
W20260928 14:55:54.408071 5352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:42013: Network error: Client connection negotiation failed: client connection to 127.14.241.252:42013: connect: Connection refused (error 111)
I20260928 14:55:54.411417 5449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:54.412835 5434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:55:54.412997 5434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/wal
uuid: "7419c23b12db417f85eeb47c233755ca"
format_stamp: "Formatted at 2026-09-28 14:55:54 on dist-test-slave-vk40"
server_key: "6027ec08912c3816cb72b95f6bf4ebfc"
server_key_iv: "bbac2fd2f622d2cdab9a2a3ea8eb83a0"
server_key_version: "encryptionkey@0"
I20260928 14:55:54.413194 5434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:54.426103 5434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:54.430313 5434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:54.430567 5434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:54.447829 5434 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.252:42013
I20260928 14:55:54.447851 5503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.252:42013 every 8 connection(s)
I20260928 14:55:54.449201 5434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
I20260928 14:55:54.453032 5504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:54.457597 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 5434
I20260928 14:55:54.457768 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790607004819300-15303-0/minicluster-data/master-2/wal/instance
I20260928 14:55:54.458261 15303 external_mini_cluster.cc:1428] Setting key 4a0dc622bb06123ce158937541dec1d6
I20260928 14:55:54.460218 5504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:55:54 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607354, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
Sep 28 14:55:54 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
I20260928 14:55:54.485744 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.464233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53768 (local address 127.14.241.254:43445)
0928 14:55:54.464480 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.464487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.467068 (+ 2581us) server_negotiation.cc:407] Connection header received
0928 14:55:54.469136 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.469143 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.469231 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.469359 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.471010 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.472064 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.473972 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.474459 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.474591 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.478721 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.478726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.478729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.478767 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.482071 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.483019 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.483024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.483028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.483206 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.483692 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.483695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.483697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.484022 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.484193 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.485181 (+ 988us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.485429 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260928 14:55:54.486960 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.463746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:43445 (local address 127.0.0.1:53768)
0928 14:55:54.466868 (+ 3122us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.466926 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:54.468854 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:54.469474 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:54.469481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:54.470308 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:54.470804 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.470832 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.472272 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.472301 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.473806 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.473820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.474756 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.474764 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.475091 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.475778 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:54.475809 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:54.478441 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:54.482336 (+ 3895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.482347 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.482359 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.482867 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.483337 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.483342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.483346 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.483589 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.484185 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:54.484200 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:54.484775 (+ 575us) client_negotiation.cc:769] Sending connection context
0928 14:55:54.485028 (+ 253us) client_negotiation.cc:240] Negotiation successful
0928 14:55:54.485643 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2867,"mutex_wait_us":56,"spinlock_wait_cycles":17152,"thread_start_us":2706,"threads_started":1}
I20260928 14:55:54.486979 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.460202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58364 (local address 127.14.241.252:42013)
0928 14:55:54.466429 (+ 6227us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.466444 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.467063 (+ 619us) server_negotiation.cc:407] Connection header received
0928 14:55:54.468868 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.468902 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.469230 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.469431 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.471234 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.472877 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.474410 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.475218 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.475312 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.479447 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.479460 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.479477 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.479517 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.483062 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.483982 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.483991 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.483994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.484149 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.484664 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.484668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.484670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.485011 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.485368 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.485761 (+ 393us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.486013 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":5975,"thread_start_us":5693,"threads_started":1}
I20260928 14:55:54.488554 5504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:55:54 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607354, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
Sep 28 14:55:54 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:54.503146 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.489427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:37305 (local address 127.0.0.1:51150)
0928 14:55:54.489727 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.489742 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:54.489847 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:54.490297 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:54.490302 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:54.490790 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:54.490926 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.490934 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.492196 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.492202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.493437 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.493449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.494341 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.494345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.494491 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.495100 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:54.495123 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:54.497656 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:54.500751 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.500757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.500761 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.501220 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.501580 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.501585 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.501589 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.501726 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.502225 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:54.502229 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:54.502389 (+ 160us) client_negotiation.cc:769] Sending connection context
0928 14:55:54.502507 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 14:55:54.502782 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260928 14:55:54.503167 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.489429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.14.241.253:37305)
0928 14:55:54.489607 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.489613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.489812 (+ 199us) server_negotiation.cc:407] Connection header received
0928 14:55:54.490063 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.490069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.490153 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.490257 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.491070 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.492055 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.493566 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.494204 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.494344 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.497874 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.497879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.497882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.497918 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.500620 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.501337 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.501342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.501344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.501475 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.501833 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.501836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.501838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.502112 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.502242 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.502615 (+ 373us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.502827 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:55:54.504128 5504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:55:54.504725 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.490190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53770 (local address 127.14.241.254:43445)
0928 14:55:54.490354 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.490359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.490375 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:55:54.490424 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.490428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.490497 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.490596 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.491909 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.492983 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.494489 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.495024 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.495149 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.499556 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.499562 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.499565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.499599 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.502060 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.502670 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.502681 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.502684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.502807 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.503141 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.503145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.503147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.503685 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.503948 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.504235 (+ 287us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.504477 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Sep 28 14:55:54 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607354, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
Sep 28 14:55:54 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
I20260928 14:55:54.518153 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.504941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58368)
0928 14:55:54.505125 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.505153 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:54.505233 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:54.505816 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:54.505821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:54.506200 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:54.506336 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.506348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.507232 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.507237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.508556 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.508574 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.509340 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.509344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.509486 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.510055 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:54.510074 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:54.512823 (+ 2749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:54.515687 (+ 2864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.515715 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.515718 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.516109 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.516506 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.516509 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.516511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.516643 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.517139 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:54.517143 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:54.517311 (+ 168us) client_negotiation.cc:769] Sending connection context
0928 14:55:54.517509 (+ 198us) client_negotiation.cc:240] Negotiation successful
0928 14:55:54.517779 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 14:55:54.518826 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.505124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.14.241.252:42013)
0928 14:55:54.505564 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.505570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.505587 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:55:54.505633 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.505638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.505707 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.505797 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.506458 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.507125 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.508697 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.509186 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.509324 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.512995 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.513000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.513003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.513032 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.515542 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.516212 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.516218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.516221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.516409 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.516769 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.516774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.516777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.517035 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.517180 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.517606 (+ 426us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.518526 (+ 920us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":712,"server-negotiator.queue_time_us":294}
I20260928 14:55:54.522624 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.506580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51154 (local address 127.14.241.253:37305)
0928 14:55:54.506734 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.506738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.507494 (+ 756us) server_negotiation.cc:407] Connection header received
0928 14:55:54.507609 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.507614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.507694 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.507790 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.509258 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.510160 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.511940 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.512521 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.512619 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.517000 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.517006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.517009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.517043 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.519876 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.520765 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.520770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.520772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.520944 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.521335 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.521338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.521340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.521605 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.521812 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.522110 (+ 298us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.522318 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 14:55:54.524544 5504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca: Bootstrap starting.
I20260928 14:55:54.527366 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:54.528235 5504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:54.529570 5504 log.cc:824] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:54.532413 5504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca: No bootstrap required, opened a new log
I20260928 14:55:54.530711 5327 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f60590b979bf4055b858e74d0fa65598"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53770
I20260928 14:55:54.539474 5504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } }
I20260928 14:55:54.539824 5504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7419c23b12db417f85eeb47c233755ca, State: Initialized, Role: FOLLOWER
I20260928 14:55:54.540266 5504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } }
I20260928 14:55:54.541234 5510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } } }
I20260928 14:55:54.541661 5510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:54.542222 5504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [sys.catalog]: configured and running, proceeding with master startup.
W20260928 14:55:54.551297 5521 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:55:54.551455 5521 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:55:54.549199 5434 master_runner.cc:186] Error getting master registration for 127.14.241.254:43445: OK, instance_id { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" instance_seqno: 1790607353451132 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:55:54.560202 5434 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:55:54.585302 5352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } attempt: 4
Sep 28 14:55:54 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607353, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
I20260928 14:55:54.600895 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.586019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58376)
0928 14:55:54.586299 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.586321 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:54.586453 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:54.586872 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:54.586878 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:54.587377 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:54.587605 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.587618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.589096 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.589102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.590565 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.590582 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.591554 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.591560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.591787 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.592455 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:54.592479 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:54.595105 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:54.598393 (+ 3288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.598399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.598404 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.598790 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.599140 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.599146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.599150 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.599285 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.599905 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:54.599910 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:54.600062 (+ 152us) client_negotiation.cc:769] Sending connection context
0928 14:55:54.600192 (+ 130us) client_negotiation.cc:240] Negotiation successful
0928 14:55:54.600504 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260928 14:55:54.600942 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.586297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58376 (local address 127.14.241.252:42013)
0928 14:55:54.586454 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.586459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.586478 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:55:54.586599 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.586618 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.586720 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.586821 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.587759 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.588916 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.590712 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.591394 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.591517 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.595305 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.595312 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.595317 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.595357 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.598258 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.598905 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.598911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.598913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.599036 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.599388 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.599391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.599393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.599779 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.599911 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.600371 (+ 460us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.600584 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:55:54.605266 5352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598: Bootstrap starting.
I20260928 14:55:54.608487 5352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:54.609506 5352 log.cc:824] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:54.612141 5352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598: No bootstrap required, opened a new log
I20260928 14:55:54.617412 5352 raft_consensus.cc:348] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } }
I20260928 14:55:54.617746 5352 raft_consensus.cc:729] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f60590b979bf4055b858e74d0fa65598, State: Initialized, Role: FOLLOWER
I20260928 14:55:54.618175 5352 consensus_queue.cc:260] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } }
I20260928 14:55:54.619135 5524 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } } }
I20260928 14:55:54.619486 5524 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:54.620252 5352 sys_catalog.cc:564] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:54.620430 5327 raft_consensus.cc:482] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 14:55:54.620671 5327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:54.624380 5327 raft_consensus.cc:504] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } }
I20260928 14:55:54.628311 5477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f60590b979bf4055b858e74d0fa65598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7419c23b12db417f85eeb47c233755ca"
I20260928 14:55:54.628840 5477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:54.632714 5477 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f60590b979bf4055b858e74d0fa65598 in term 1.
I20260928 14:55:54.633862 5303 leader_election.cc:302] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7419c23b12db417f85eeb47c233755ca, f60590b979bf4055b858e74d0fa65598; no voters:
I20260928 14:55:54.634346 5524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:54.640486 5327 leader_election.cc:288] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [CANDIDATE]: Term 1 election: Requested vote from peers 0092a1ccec03481095e99083594a7950 (127.14.241.253:37305), 7419c23b12db417f85eeb47c233755ca (127.14.241.252:42013)
I20260928 14:55:54.635090 5285 master_runner.cc:186] Error getting master registration for 127.14.241.253:37305: OK, instance_id { permanent_uuid: "0092a1ccec03481095e99083594a7950" instance_seqno: 1790607353957831 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:55:54.652493 5285 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:55:54.661569 5524 raft_consensus.cc:686] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [term 1 LEADER]: Becoming Leader. State: Replica: f60590b979bf4055b858e74d0fa65598, State: Running, Role: LEADER
I20260928 14:55:54.662315 5524 consensus_queue.cc:237] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } }
I20260928 14:55:54.674444 5538 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f60590b979bf4055b858e74d0fa65598. Latest consensus state: current_term: 1 leader_uuid: "f60590b979bf4055b858e74d0fa65598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } } }
I20260928 14:55:54.675316 5538 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:54.682279 5427 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } attempt: 5
W20260928 14:55:54.682667 5455 tablet.cc:2369] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:55:54.686549 5536 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:55:54.686741 5536 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260928 14:55:54.692233 5306 tablet.cc:2369] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:55:54.693905 5547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:54.696306 5547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:54.706403 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.660019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51160 (local address 127.14.241.253:37305)
0928 14:55:54.660234 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.660241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.669275 (+ 9034us) server_negotiation.cc:407] Connection header received
0928 14:55:54.669481 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.669488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.669584 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.669702 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.675630 (+ 5928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.676836 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.689983 (+ 13147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.690661 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.690817 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.698333 (+ 7516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.698341 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.698346 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.698392 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.702154 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.703437 (+ 1283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.703444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.703448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.703728 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.704481 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.704486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.704509 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.704956 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.705220 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.705805 (+ 585us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.706058 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Sep 28 14:55:54 dist-test-slave-vk40 krb5kdc[5272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607353, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
I20260928 14:55:54.715065 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.660586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53780 (local address 127.14.241.254:43445)
0928 14:55:54.660779 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.660786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.670907 (+ 10121us) server_negotiation.cc:407] Connection header received
0928 14:55:54.671144 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.671150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.671228 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.671299 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.680858 (+ 9559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.682034 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.691011 (+ 8977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.691727 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.691878 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.707979 (+ 16101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.707987 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.707992 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.708038 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.711696 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.712574 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.712581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.712585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.712754 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.713184 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.713190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.713193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.713546 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.713801 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.714445 (+ 644us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.714705 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:55:54.716132 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.669942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.14.241.252:42013)
0928 14:55:54.670111 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.670116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.679616 (+ 9500us) server_negotiation.cc:407] Connection header received
0928 14:55:54.679903 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.679909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.679998 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.680062 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.690608 (+ 10546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.691825 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.696517 (+ 4692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.697265 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.697374 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.709142 (+ 11768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.709149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.709154 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.709197 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.712964 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.713805 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.713812 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.713816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.713981 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.714429 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.714435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.714438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.714793 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.715032 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.715534 (+ 502us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.715771 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 14:55:54.725001 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.682960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:42013 (local address 127.0.0.1:58390)
0928 14:55:54.683323 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:54.683343 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:54.683447 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:54.705143 (+ 21696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:54.705149 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:54.705631 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:54.705841 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.705853 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.708231 (+ 2378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.708237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.709576 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:54.709591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.711277 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.711285 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:54.711525 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.712122 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:54.712143 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:54.715080 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:54.721503 (+ 6423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.721510 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.721514 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.722044 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.722720 (+ 676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:54.722726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:54.722730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:54.722928 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:54.723976 (+ 1048us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:54.723982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:54.724143 (+ 161us) client_negotiation.cc:769] Sending connection context
0928 14:55:54.724320 (+ 177us) client_negotiation.cc:240] Negotiation successful
0928 14:55:54.724627 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260928 14:55:54.726821 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:54.683655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.14.241.252:42013)
0928 14:55:54.704804 (+ 21149us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.704810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.704834 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:55:54.704893 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.704898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.704977 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.705083 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.706091 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.708020 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.709882 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:54.711025 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:54.715482 (+ 4457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:54.715871 (+ 389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:54.715879 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:54.715884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:54.715948 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:54.721263 (+ 5315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.722295 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.722301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.722304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.722556 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:54.723163 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:54.723171 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:54.723174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:54.723789 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:54.725485 (+ 1696us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:54.725692 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 14:55:54.726085 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20996,"thread_start_us":20759,"threads_started":1}
I20260928 14:55:54.732589 5427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950: Bootstrap starting.
I20260928 14:55:54.735729 5477 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [term 1 FOLLOWER]: Refusing update from remote peer f60590b979bf4055b858e74d0fa65598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:54.736994 5538 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:54.746250 5427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:54.749614 5427 log.cc:824] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:54.754112 5427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950: No bootstrap required, opened a new log
I20260928 14:55:54.765517 5510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader f60590b979bf4055b858e74d0fa65598. Latest consensus state: current_term: 1 leader_uuid: "f60590b979bf4055b858e74d0fa65598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } } }
I20260928 14:55:54.765841 5510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:54.768689 5549 mvcc.cc:205] Tried to move back new op lower bound from 7334327724981866496 to 7334327724715470848. Current Snapshot: MvccSnapshot[applied={T|T < 7334327724981866496}]
I20260928 14:55:54.770318 5550 mvcc.cc:205] Tried to move back new op lower bound from 7334327724981866496 to 7334327724715470848. Current Snapshot: MvccSnapshot[applied={T|T < 7334327724981866496}]
I20260928 14:55:54.776893 5427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } }
I20260928 14:55:54.777470 5427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0092a1ccec03481095e99083594a7950, State: Initialized, Role: FOLLOWER
I20260928 14:55:54.778066 5427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } }
I20260928 14:55:54.785204 5555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } } }
I20260928 14:55:54.787590 5555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:54.787459 5427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:54.787871 5401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:54.788483 5402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f60590b979bf4055b858e74d0fa65598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0092a1ccec03481095e99083594a7950"
I20260928 14:55:54.792546 5401 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [term 1 FOLLOWER]: Refusing update from remote peer f60590b979bf4055b858e74d0fa65598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:54.796532 5402 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f60590b979bf4055b858e74d0fa65598 in term 1.
I20260928 14:55:54.799701 5538 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:54.812454 5510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f60590b979bf4055b858e74d0fa65598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } } }
I20260928 14:55:54.812768 5510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:54.820583 5538 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f60590b979bf4055b858e74d0fa65598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } } }
I20260928 14:55:54.820891 5538 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:54.821777 5547 catalog_manager.cc:1347] Generated new cluster ID: b625bcc492da4676950456b93d1992f1
I20260928 14:55:54.821892 5547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:54.833459 5555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f60590b979bf4055b858e74d0fa65598. Latest consensus state: current_term: 1 leader_uuid: "f60590b979bf4055b858e74d0fa65598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } } }
I20260928 14:55:54.833797 5555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:54.834354 5565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f60590b979bf4055b858e74d0fa65598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } } }
I20260928 14:55:54.835718 5565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:54.843348 5570 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 14:55:54.847039 5524 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f60590b979bf4055b858e74d0fa65598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f60590b979bf4055b858e74d0fa65598" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 43445 } } peers { permanent_uuid: "0092a1ccec03481095e99083594a7950" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 37305 } } peers { permanent_uuid: "7419c23b12db417f85eeb47c233755ca" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 42013 } } }
I20260928 14:55:54.847344 5524 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:54.856950 5547 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:54.861189 5547 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:54.875425 5547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f60590b979bf4055b858e74d0fa65598: Generated new TSK 0
I20260928 14:55:54.876680 5547 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:55:54.951152 5356 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:54.934847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53790 (local address 127.14.241.254:43445)
0928 14:55:54.935029 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.935035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.947638 (+ 12603us) server_negotiation.cc:407] Connection header received
0928 14:55:54.949383 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.949391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.949489 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.949574 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.950864 (+ 1290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53790: BlockingRecv error: recv got EOF from 127.0.0.1:53790 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260928 14:55:54.955802 5432 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:54.934686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.14.241.253:37305)
0928 14:55:54.934887 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.934893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.952686 (+ 17793us) server_negotiation.cc:407] Connection header received
0928 14:55:54.954267 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.954274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.954385 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.954465 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.955542 (+ 1077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51176: BlockingRecv error: recv got EOF from 127.0.0.1:51176 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260928 14:55:54.958063 5548 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:54.948368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58406 (local address 127.14.241.252:42013)
0928 14:55:54.948566 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.948573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.953823 (+ 5250us) server_negotiation.cc:407] Connection header received
0928 14:55:54.956498 (+ 2675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.956505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.956622 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.956692 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.957810 (+ 1118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58406: BlockingRecv error: recv got EOF from 127.0.0.1:58406 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260928 14:55:54.975674 5432 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:54.963652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51186 (local address 127.14.241.253:37305)
0928 14:55:54.963827 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.963833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.973659 (+ 9826us) server_negotiation.cc:407] Connection header received
0928 14:55:54.974158 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.974164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.974289 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.974358 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.975368 (+ 1010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51186: BlockingRecv error: recv got EOF from 127.0.0.1:51186 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260928 14:55:54.998203 5432 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:54.993868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.14.241.253:37305)
0928 14:55:54.994063 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:54.994070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:54.996833 (+ 2763us) server_negotiation.cc:407] Connection header received
0928 14:55:54.997026 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:54.997030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:54.997101 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:54.997194 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:54.997974 (+ 780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51188: BlockingRecv error: recv got EOF from 127.0.0.1:51188 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260928 14:55:55.004784 5432 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:55.002846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51190 (local address 127.14.241.253:37305)
0928 14:55:55.003033 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:55.003040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:55.003501 (+ 461us) server_negotiation.cc:407] Connection header received
0928 14:55:55.003692 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:55.003696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:55.003780 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:55.003865 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:55.004550 (+ 685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51190: BlockingRecv error: recv got EOF from 127.0.0.1:51190 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260928 14:55:55.015981 5432 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:55.013958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.14.241.253:37305)
0928 14:55:55.014169 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:55.014176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:55.014686 (+ 510us) server_negotiation.cc:407] Connection header received
0928 14:55:55.014876 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:55.014881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:55.014966 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:55.015052 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:55.015766 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51194: BlockingRecv error: recv got EOF from 127.0.0.1:51194 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260928 14:55:55.029166 5432 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:55.027113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51206 (local address 127.14.241.253:37305)
0928 14:55:55.027333 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:55.027340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:55.027791 (+ 451us) server_negotiation.cc:407] Connection header received
0928 14:55:55.027982 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:55.027987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:55.028060 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:55.028147 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:55.028930 (+ 783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51206: BlockingRecv error: recv got EOF from 127.0.0.1:51206 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260928 14:55:55.051096 5432 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:55.049022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51212 (local address 127.14.241.253:37305)
0928 14:55:55.049223 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:55.049229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:55.049739 (+ 510us) server_negotiation.cc:407] Connection header received
0928 14:55:55.049940 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:55.049944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:55.050030 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:55.050115 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:55.050852 (+ 737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51212: BlockingRecv error: recv got EOF from 127.0.0.1:51212 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260928 14:55:55.088541 5432 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:55.086296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.14.241.253:37305)
0928 14:55:55.086503 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:55.086510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:55.087070 (+ 560us) server_negotiation.cc:407] Connection header received
0928 14:55:55.087268 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:55.087274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:55.087350 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:55.087462 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:55.088209 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51224: BlockingRecv error: recv got EOF from 127.0.0.1:51224 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260928 14:55:55.157546 5432 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:55.155781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51236 (local address 127.14.241.253:37305)
0928 14:55:55.156030 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:55.156037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:55.156200 (+ 163us) server_negotiation.cc:407] Connection header received
0928 14:55:55.156375 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:55.156382 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:55.156481 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:55.156625 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:55.157324 (+ 699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51236: BlockingRecv error: recv got EOF from 127.0.0.1:51236 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260928 14:55:55.290695 5432 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:55.288939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51252 (local address 127.14.241.253:37305)
0928 14:55:55.289125 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:55.289131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:55.289464 (+ 333us) server_negotiation.cc:407] Connection header received
0928 14:55:55.289629 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:55.289633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:55.289707 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:55.289799 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:55.290470 (+ 671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51252: BlockingRecv error: recv got EOF from 127.0.0.1:51252 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260928 14:55:55.551985 5432 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:55.550200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.14.241.253:37305)
0928 14:55:55.550428 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:55.550434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:55.550629 (+ 195us) server_negotiation.cc:407] Connection header received
0928 14:55:55.550861 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:55.550865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:55.550940 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:55.551028 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:55.551764 (+ 736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51268: BlockingRecv error: recv got EOF from 127.0.0.1:51268 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 14:55:55.553565 5521 catalog_manager.cc:1259] Loaded cluster ID: b625bcc492da4676950456b93d1992f1
I20260928 14:55:55.553653 5521 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca: loading cluster ID for follower catalog manager: success
I20260928 14:55:55.556093 5521 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca: acquiring CA information for follower catalog manager: success
I20260928 14:55:55.557855 5521 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7419c23b12db417f85eeb47c233755ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:55:55.847859 5570 catalog_manager.cc:1259] Loaded cluster ID: b625bcc492da4676950456b93d1992f1
I20260928 14:55:55.847991 5570 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950: loading cluster ID for follower catalog manager: success
I20260928 14:55:55.850850 5570 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950: acquiring CA information for follower catalog manager: success
I20260928 14:55:55.852752 5570 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0092a1ccec03481095e99083594a7950: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:55:56.084689 5637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:56.077184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51274 (local address 127.14.241.253:37305)
0928 14:55:56.077866 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:56.077873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:56.078195 (+ 322us) server_negotiation.cc:407] Connection header received
0928 14:55:56.078456 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:56.078460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:56.078654 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:56.078772 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:55:56.079340 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:56.080232 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:56.082211 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:56.082680 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:56.082796 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:56.083413 (+ 617us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:55:56.083990 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:55:56.084104 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:56.084267 (+ 163us) server_negotiation.cc:299] Negotiation successful
0928 14:55:56.084373 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":336,"threads_started":1}
I20260928 14:55:56.097921 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:56.091790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58412 (local address 127.14.241.252:42013)
0928 14:55:56.092743 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:56.092748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:56.092764 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:55:56.092860 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:56.092864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:56.093006 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:56.093160 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:55:56.093611 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:56.094358 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:56.095736 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:56.096145 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:56.096268 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:56.096778 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:55:56.097307 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:55:56.097453 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:56.097547 (+ 94us) server_negotiation.cc:299] Negotiation successful
0928 14:55:56.097637 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":754,"threads_started":1}
I20260928 14:55:56.112730 5637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:56.104309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51284 (local address 127.14.241.253:37305)
0928 14:55:56.104488 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:56.104493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:56.105369 (+ 876us) server_negotiation.cc:407] Connection header received
0928 14:55:56.105673 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:56.105679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:56.105771 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:56.105839 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:55:56.106582 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:56.107872 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:56.110281 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:56.111088 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:56.111205 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:56.111877 (+ 672us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:55:56.112010 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:55:56.112079 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:56.112327 (+ 248us) server_negotiation.cc:299] Negotiation successful
0928 14:55:56.112412 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 14:55:56.118834 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:56.105566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58414 (local address 127.14.241.252:42013)
0928 14:55:56.105742 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:56.105747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:56.112155 (+ 6408us) server_negotiation.cc:407] Connection header received
0928 14:55:56.112471 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:56.112478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:56.112580 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:56.112652 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:55:56.113442 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:56.114522 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:56.116651 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:56.117326 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:56.117429 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:56.118047 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:55:56.118209 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:55:56.118311 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:56.118443 (+ 132us) server_negotiation.cc:299] Negotiation successful
0928 14:55:56.118524 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 14:55:56.119411 5648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:56.104026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53796 (local address 127.14.241.254:43445)
0928 14:55:56.104481 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:56.104489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:56.111810 (+ 7321us) server_negotiation.cc:407] Connection header received
0928 14:55:56.112263 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:56.112270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:56.112511 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:56.112665 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:55:56.113574 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:56.114369 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:56.116597 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:56.117265 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:56.117414 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:56.118224 (+ 810us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:55:56.118678 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:55:56.118794 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:56.119028 (+ 234us) server_negotiation.cc:299] Negotiation successful
0928 14:55:56.119123 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":468,"threads_started":1}
I20260928 14:55:56.125437 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 5285
I20260928 14:55:56.126856 5347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:56.239733 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 5285
I20260928 14:55:56.257505 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 5359
I20260928 14:55:56.258673 5422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:56.365132 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 5359
I20260928 14:55:56.382282 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 5434
I20260928 14:55:56.383390 5498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:55:56.495430 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 5434
2026-09-28T14:55:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4757 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:55:56 dist-test-slave-vk40 krb5kdc[5657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:55:56 dist-test-slave-vk40 krb5kdc[5657](info): set up 2 sockets
Sep 28 14:55:56 dist-test-slave-vk40 krb5kdc[5657](info): commencing operation
krb5kdc: starting...
W20260928 14:55:57.523401 15303 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/test-user.keytab.
Sep 28 14:55:57 dist-test-slave-vk40 krb5kdc[5657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:55:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:55:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
Entry for principal kudu/127.14.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab.
I20260928 14:55:57.774989 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38733
--webserver_interface=127.14.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:38733,127.14.241.253:33255,127.14.241.252:45353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:55:58.094955 5670 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:55:58.095273 5670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:58.095343 5670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:58.095424 5670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:58.102553 5670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:58.102646 5670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:58.102710 5670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:58.102759 5670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:58.102825 5670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:58.112593 5670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:38733,127.14.241.253:33255,127.14.241.252:45353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.254.keytab
--principal=kudu/127.14.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.254:38733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:58.113814 5670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:58.115090 5670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:58.126537 5675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:58.126494 5676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:58.126510 5678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:58.127665 5670 server_base.cc:1034] running on GCE node
I20260928 14:55:58.128453 5670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:58.129915 5670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:58.131420 5670 hybrid_clock.cc:648] HybridClock initialized: now 1790607358131324 us; error 59 us; skew 500 ppm
I20260928 14:55:58.131842 5670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:58 dist-test-slave-vk40 krb5kdc[5657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607358, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:58.140625 5670 init.cc:377] Logged in from keytab as kudu/127.14.241.254@KRBTEST.COM (short username kudu)
I20260928 14:55:58.142269 5670 webserver.cc:460] Webserver started at http://127.14.241.254:36467/ using document root <none> and password file <none>
I20260928 14:55:58.142995 5670 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:58.143101 5670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:58.143427 5670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:58.146541 5670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/data/instance:
uuid: "44c9c4a027b44dc58545fd5343e803a6"
format_stamp: "Formatted at 2026-09-28 14:55:58 on dist-test-slave-vk40"
server_key: "bd9af5775413fb66a42fe3271fc04f7d"
server_key_iv: "8573e4a18eab6a58320f352e99fb4016"
server_key_version: "encryptionkey@0"
I20260928 14:55:58.147365 5670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/wal/instance:
uuid: "44c9c4a027b44dc58545fd5343e803a6"
format_stamp: "Formatted at 2026-09-28 14:55:58 on dist-test-slave-vk40"
server_key: "bd9af5775413fb66a42fe3271fc04f7d"
server_key_iv: "8573e4a18eab6a58320f352e99fb4016"
server_key_version: "encryptionkey@0"
I20260928 14:55:58.152657 5670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260928 14:55:58.156152 5685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:58.157502 5670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:55:58.157670 5670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/wal
uuid: "44c9c4a027b44dc58545fd5343e803a6"
format_stamp: "Formatted at 2026-09-28 14:55:58 on dist-test-slave-vk40"
server_key: "bd9af5775413fb66a42fe3271fc04f7d"
server_key_iv: "8573e4a18eab6a58320f352e99fb4016"
server_key_version: "encryptionkey@0"
I20260928 14:55:58.157877 5670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:58.170925 5670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:58.175261 5670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:58.175518 5670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:58.193482 5670 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.254:38733
I20260928 14:55:58.193504 5736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.254:38733 every 8 connection(s)
I20260928 14:55:58.195066 5670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/data/info.pb
I20260928 14:55:58.198187 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 5670
I20260928 14:55:58.198359 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-0/wal/instance
I20260928 14:55:58.198786 15303 external_mini_cluster.cc:1428] Setting key 97b0df5d7e39d14c8e05c90d35ea6557
I20260928 14:55:58.198851 5737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:58.205819 5737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:55:58 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607358, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.241.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.253@KRBTEST.COM" created.
I20260928 14:55:58.228792 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.209522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38733 (local address 127.0.0.1:49642)
0928 14:55:58.210340 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.210389 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:58.212529 (+ 2140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:58.213410 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:58.213416 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:58.214130 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:58.214479 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:58.214498 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.216034 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.216038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:58.217179 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:58.217192 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.218105 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.218111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:58.218308 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:58.218903 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:58.218926 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:58.221376 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:58.224848 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:58.224859 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:58.224870 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:58.225280 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:58.225756 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:58.225759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:58.225761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:58.225913 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:58.226824 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:58.226829 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:58.227026 (+ 197us) client_negotiation.cc:769] Sending connection context
0928 14:55:58.227238 (+ 212us) client_negotiation.cc:240] Negotiation successful
0928 14:55:58.227639 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":334,"threads_started":1}
I20260928 14:55:58.228818 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.210883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.14.241.254:38733)
0928 14:55:58.211570 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:58.211581 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:58.211640 (+ 59us) server_negotiation.cc:407] Connection header received
0928 14:55:58.212906 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:58.212912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:58.213175 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:58.213370 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:58.214682 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.215887 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.217321 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.217884 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.218061 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:58.221649 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:58.221657 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:58.221670 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:58.221706 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:58.224703 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:58.225437 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:58.225446 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:58.225449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:58.225647 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:58.226013 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:58.226018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:58.226022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:58.226691 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:58.226899 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:58.227405 (+ 506us) server_negotiation.cc:299] Negotiation successful
0928 14:55:58.227809 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":490,"thread_start_us":314,"threads_started":1}
I20260928 14:55:58.232035 5737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:55:58.233304 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.232852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38386)
0928 14:55:58.233046 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.233165 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260928 14:55:58.233642 5689 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
I20260928 14:55:58.235273 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.234866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38388)
0928 14:55:58.235030 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.235117 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260928 14:55:58.235486 5737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:33255: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab.
Entry for principal kudu/127.14.241.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab.
I20260928 14:55:58.257555 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.253:33255
--webserver_interface=127.14.241.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab
--principal=kudu/127.14.241.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:38733,127.14.241.253:33255,127.14.241.252:45353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:55:58.284832 5737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } attempt: 1
I20260928 14:55:58.286118 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.285595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38394)
0928 14:55:58.285818 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.285952 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260928 14:55:58.287112 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.286728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38396)
0928 14:55:58.286888 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.286969 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260928 14:55:58.287313 5737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:33255: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
I20260928 14:55:58.355566 5737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } attempt: 2
I20260928 14:55:58.356805 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.356243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38406)
0928 14:55:58.356503 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.356620 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260928 14:55:58.357836 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.357437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38422)
0928 14:55:58.357576 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.357674 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 14:55:58.358018 5737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:33255: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
I20260928 14:55:58.449299 5737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } attempt: 3
I20260928 14:55:58.450511 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.450008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38438)
0928 14:55:58.450234 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.450347 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260928 14:55:58.451618 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.451214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38452)
0928 14:55:58.451369 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.451452 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260928 14:55:58.451824 5737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:33255: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
W20260928 14:55:58.577795 5744 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:55:58.578109 5744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:58.578207 5744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:58.578311 5744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:58.585515 5744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:58.585604 5744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:58.585742 5744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:58.585867 5744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:58.585958 5744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:58.595082 5737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } attempt: 4
I20260928 14:55:58.596179 5744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:38733,127.14.241.253:33255,127.14.241.252:45353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.253.keytab
--principal=kudu/127.14.241.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.253:33255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.241.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:58.596352 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.595717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38456)
0928 14:55:58.596053 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.596161 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260928 14:55:58.597319 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.597014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38470)
0928 14:55:58.597128 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.597207 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260928 14:55:58.597491 5737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.253:33255: Network error: Client connection negotiation failed: client connection to 127.14.241.253:33255: connect: Connection refused (error 111)
I20260928 14:55:58.597754 5744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:58.598990 5744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:55:58.612617 5749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:58.612520 5752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:58.612551 5750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:58.613436 5744 server_base.cc:1034] running on GCE node
I20260928 14:55:58.614156 5744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:58.615490 5744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:58.616693 5744 hybrid_clock.cc:648] HybridClock initialized: now 1790607358616628 us; error 34 us; skew 500 ppm
I20260928 14:55:58.617005 5744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:58 dist-test-slave-vk40 krb5kdc[5657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607358, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:58.619889 5744 init.cc:377] Logged in from keytab as kudu/127.14.241.253@KRBTEST.COM (short username kudu)
I20260928 14:55:58.621260 5744 webserver.cc:460] Webserver started at http://127.14.241.253:44313/ using document root <none> and password file <none>
I20260928 14:55:58.621908 5744 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:58.621989 5744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:58.622241 5744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:58.625149 5744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/data/instance:
uuid: "4a903636b5e5422e9c140129df6e45f0"
format_stamp: "Formatted at 2026-09-28 14:55:58 on dist-test-slave-vk40"
server_key: "436a507e70989c401fe7f9614b4ddd2d"
server_key_iv: "b5874967314425d7885279d0f3ad085d"
server_key_version: "encryptionkey@0"
I20260928 14:55:58.625943 5744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/wal/instance:
uuid: "4a903636b5e5422e9c140129df6e45f0"
format_stamp: "Formatted at 2026-09-28 14:55:58 on dist-test-slave-vk40"
server_key: "436a507e70989c401fe7f9614b4ddd2d"
server_key_iv: "b5874967314425d7885279d0f3ad085d"
server_key_version: "encryptionkey@0"
I20260928 14:55:58.630939 5744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 14:55:58.634243 5759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:58.635545 5744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:55:58.635702 5744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/wal
uuid: "4a903636b5e5422e9c140129df6e45f0"
format_stamp: "Formatted at 2026-09-28 14:55:58 on dist-test-slave-vk40"
server_key: "436a507e70989c401fe7f9614b4ddd2d"
server_key_iv: "b5874967314425d7885279d0f3ad085d"
server_key_version: "encryptionkey@0"
I20260928 14:55:58.635895 5744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:58.654229 5744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:58.658449 5744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:58.658710 5744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:58.676011 5744 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.253:33255
I20260928 14:55:58.676031 5813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.253:33255 every 8 connection(s)
I20260928 14:55:58.677359 5744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/data/info.pb
I20260928 14:55:58.681084 5815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:58.681396 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 5744
I20260928 14:55:58.681574 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-1/wal/instance
I20260928 14:55:58.682099 15303 external_mini_cluster.cc:1428] Setting key 69407a545ab2b66a35cdd34b6167f707
I20260928 14:55:58.689865 5815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:55:58 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607358, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.241.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.241.252@KRBTEST.COM" created.
I20260928 14:55:58.710017 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.693044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.14.241.254:38733)
0928 14:55:58.693257 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:58.693263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:58.694510 (+ 1247us) server_negotiation.cc:407] Connection header received
0928 14:55:58.696489 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:58.696494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:58.696576 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:58.696661 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:58.698159 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.698842 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.700371 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.700792 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.700904 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:58.704309 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:58.704316 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:58.704321 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:58.704360 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:58.706867 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:58.707643 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:58.707647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:58.707649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:58.707778 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:58.708195 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:58.708200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:58.708203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:58.708531 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:58.708664 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:58.709443 (+ 779us) server_negotiation.cc:299] Negotiation successful
0928 14:55:58.709684 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:55:58.710969 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.693564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38733 (local address 127.0.0.1:49650)
0928 14:55:58.694348 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.694387 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:58.696174 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:58.696910 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:58.696918 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:58.697446 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:58.697999 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:58.698015 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.698990 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.698994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:58.700193 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:58.700206 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.701011 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.701019 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:58.701238 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:58.701865 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:58.701888 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:58.704054 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:58.707008 (+ 2954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:58.707017 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:58.707035 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:58.707525 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:58.707881 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:58.707886 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:58.707890 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:58.708099 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:58.708659 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:58.708668 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:58.709029 (+ 361us) client_negotiation.cc:769] Sending connection context
0928 14:55:58.709336 (+ 307us) client_negotiation.cc:240] Negotiation successful
0928 14:55:58.709856 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":325,"threads_started":1}
I20260928 14:55:58.712817 5815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:55:58 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607358, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
I20260928 14:55:58.726497 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.713520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38486)
0928 14:55:58.713681 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.713701 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:58.713806 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:58.714784 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:58.714789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:58.715192 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:58.715326 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:58.715335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.716701 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.716706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:58.717535 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:58.717546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.718307 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.718312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:58.718483 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:58.718985 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:58.719003 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:58.721096 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:58.724064 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:58.724069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:58.724072 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:58.724547 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:58.724883 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:58.724887 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:58.724890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:58.725041 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:58.725591 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:58.725595 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:58.725745 (+ 150us) client_negotiation.cc:769] Sending connection context
0928 14:55:58.725881 (+ 136us) client_negotiation.cc:240] Negotiation successful
0928 14:55:58.726155 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260928 14:55:58.726569 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.713772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38486 (local address 127.14.241.253:33255)
0928 14:55:58.714399 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:58.714405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:58.714429 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:55:58.714473 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:58.714478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:58.714648 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:58.714785 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:58.715478 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.716584 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.717654 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.718174 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.718279 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:58.721303 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:58.721315 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:58.721330 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:58.721358 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:58.723929 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:58.724658 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:58.724665 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:58.724667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:58.724789 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:58.725138 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:58.725141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:58.725143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:58.725470 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:58.725621 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:58.726013 (+ 392us) server_negotiation.cc:299] Negotiation successful
0928 14:55:58.726267 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":486,"thread_start_us":341,"threads_started":1}
I20260928 14:55:58.729241 5815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:55:58.730367 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.729944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49488)
0928 14:55:58.730118 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.730217 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260928 14:55:58.730618 5760 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
I20260928 14:55:58.732038 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.731712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49494)
0928 14:55:58.731870 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.731942 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 14:55:58.732246 5815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:45353: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.241.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab.
Entry for principal kudu/127.14.241.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab.
I20260928 14:55:58.740314 15303 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKMueFh/build/asan/bin/kudu
/tmp/dist-test-taskKMueFh/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.252:45353
--webserver_interface=127.14.241.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab
--principal=kudu/127.14.241.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.241.212:36943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.241.254:38733,127.14.241.253:33255,127.14.241.252:45353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:55:58.781553 5815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } attempt: 1
I20260928 14:55:58.782632 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.782183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49504)
0928 14:55:58.782379 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.782485 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260928 14:55:58.783560 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.783216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49506)
0928 14:55:58.783379 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.783453 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 14:55:58.783749 5815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:45353: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
I20260928 14:55:58.851931 5815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } attempt: 2
I20260928 14:55:58.852954 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.852531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49514)
0928 14:55:58.852715 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.852806 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260928 14:55:58.853828 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.853512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49522)
0928 14:55:58.853644 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.853707 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260928 14:55:58.854017 5815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:45353: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
I20260928 14:55:58.896703 5737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } attempt: 5
Sep 28 14:55:58 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607358, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
I20260928 14:55:58.910945 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.897452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38494)
0928 14:55:58.897660 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.897681 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:58.897833 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:58.898179 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:58.898185 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:58.898577 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:58.898801 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:58.898811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.899792 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.899796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:58.900698 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:58.900710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.901455 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.901461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:58.901655 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:58.902218 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:58.902236 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:58.904578 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:58.908438 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:58.908442 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:58.908445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:58.908877 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:58.909354 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:58.909358 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:58.909360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:58.909527 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:58.910131 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:58.910134 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:58.910288 (+ 154us) client_negotiation.cc:769] Sending connection context
0928 14:55:58.910401 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 14:55:58.910683 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260928 14:55:58.911125 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.897497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.14.241.253:33255)
0928 14:55:58.897684 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:58.897690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:58.897759 (+ 69us) server_negotiation.cc:407] Connection header received
0928 14:55:58.897970 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:58.897974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:58.898041 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:58.898125 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:58.898954 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.899655 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.900852 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:58.901302 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:58.901409 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:58.904824 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:58.904832 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:58.904837 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:58.904887 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:58.908233 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:58.909010 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:58.909016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:58.909020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:58.909194 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:58.909642 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:58.909648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:58.909651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:58.910026 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:58.910173 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:58.910578 (+ 405us) server_negotiation.cc:299] Negotiation successful
0928 14:55:58.910831 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260928 14:55:58.912036 5737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:55:58.913074 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.912718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49536)
0928 14:55:58.912868 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.912944 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260928 14:55:58.913897 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.913622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49542)
0928 14:55:58.913726 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.913810 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 14:55:58.914098 5737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:45353: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
I20260928 14:55:58.945201 5815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } attempt: 3
I20260928 14:55:58.946265 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.945801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49548)
0928 14:55:58.945993 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.946089 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260928 14:55:58.947369 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.946973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49560)
0928 14:55:58.947182 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.947258 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260928 14:55:58.947573 5815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:45353: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
I20260928 14:55:58.965305 5737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } attempt: 1
I20260928 14:55:58.966356 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.965939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49574)
0928 14:55:58.966110 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.966198 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260928 14:55:58.967231 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:58.966958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49578)
0928 14:55:58.967070 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:58.967135 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260928 14:55:58.967417 5737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:45353: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
I20260928 14:55:59.035609 5737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } attempt: 2
I20260928 14:55:59.036809 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.036339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49584)
0928 14:55:59.036544 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:59.036634 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260928 14:55:59.037719 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.037440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49586)
0928 14:55:59.037556 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:59.037619 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260928 14:55:59.037921 5737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:45353: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
W20260928 14:55:59.062738 5822 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:55:59.063050 5822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:55:59.063118 5822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:55:59.063196 5822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:55:59.070475 5822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:55:59.070565 5822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:55:59.070621 5822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:55:59.070668 5822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:55:59.070726 5822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:55:59.080576 5822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.241.212:36943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.241.254:38733,127.14.241.253:33255,127.14.241.252:45353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/krb5kdc/kudu_127.14.241.252.keytab
--principal=kudu/127.14.241.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.241.252:45353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.241.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 28 Sep 2026 14:00:13 UTC on ccycloud.quasar-sekvba.root.comops.site
build id 4677370
ASAN enabled
I20260928 14:55:59.081807 5822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:55:59.083118 5822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:55:59.090782 5815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } attempt: 4
I20260928 14:55:59.091957 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.091395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49596)
0928 14:55:59.091626 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:59.091734 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260928 14:55:59.092960 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.092595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49602)
0928 14:55:59.092738 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:59.092834 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 14:55:59.093156 5815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:45353: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
W20260928 14:55:59.094977 5829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:59.095014 5827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:55:59.094942 5831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:55:59.095747 5822 server_base.cc:1034] running on GCE node
I20260928 14:55:59.096505 5822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:55:59.097852 5822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:55:59.099046 5822 hybrid_clock.cc:648] HybridClock initialized: now 1790607359098975 us; error 48 us; skew 500 ppm
I20260928 14:55:59.099373 5822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 28 14:55:59 dist-test-slave-vk40 krb5kdc[5657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607359, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:55:59.102377 5822 init.cc:377] Logged in from keytab as kudu/127.14.241.252@KRBTEST.COM (short username kudu)
I20260928 14:55:59.103781 5822 webserver.cc:460] Webserver started at http://127.14.241.252:39393/ using document root <none> and password file <none>
I20260928 14:55:59.104475 5822 fs_manager.cc:359] Metadata directory not provided
I20260928 14:55:59.104563 5822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:55:59.104825 5822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:55:59.107636 5822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/data/instance:
uuid: "8ab76abb58484dd591a7f740007f700d"
format_stamp: "Formatted at 2026-09-28 14:55:59 on dist-test-slave-vk40"
server_key: "fb8a78d1088a12373aa42885f486b407"
server_key_iv: "cd437ad3452133893d0ee9fc16573cbc"
server_key_version: "encryptionkey@0"
I20260928 14:55:59.108381 5822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/wal/instance:
uuid: "8ab76abb58484dd591a7f740007f700d"
format_stamp: "Formatted at 2026-09-28 14:55:59 on dist-test-slave-vk40"
server_key: "fb8a78d1088a12373aa42885f486b407"
server_key_iv: "cd437ad3452133893d0ee9fc16573cbc"
server_key_version: "encryptionkey@0"
I20260928 14:55:59.115775 5822 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260928 14:55:59.119678 5838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:55:59.121068 5822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 14:55:59.121236 5822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/data,/tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/wal
uuid: "8ab76abb58484dd591a7f740007f700d"
format_stamp: "Formatted at 2026-09-28 14:55:59 on dist-test-slave-vk40"
server_key: "fb8a78d1088a12373aa42885f486b407"
server_key_iv: "cd437ad3452133893d0ee9fc16573cbc"
server_key_version: "encryptionkey@0"
I20260928 14:55:59.121454 5822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:55:59.144102 5737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } attempt: 3
I20260928 14:55:59.145221 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.144778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49612)
0928 14:55:59.144960 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:59.145052 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260928 14:55:59.146183 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.145812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49620)
0928 14:55:59.146010 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:59.146078 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260928 14:55:59.146433 5737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.241.252:45353: Network error: Client connection negotiation failed: client connection to 127.14.241.252:45353: connect: Connection refused (error 111)
I20260928 14:55:59.157223 5822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:55:59.160961 5822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:55:59.161203 5822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:55:59.177944 5822 rpc_server.cc:307] RPC server started. Bound to: 127.14.241.252:45353
I20260928 14:55:59.177963 5891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.241.252:45353 every 8 connection(s)
I20260928 14:55:59.179270 5822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/data/info.pb
I20260928 14:55:59.182962 5892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:55:59.183416 15303 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKMueFh/build/asan/bin/kudu as pid 5822
I20260928 14:55:59.183565 15303 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKMueFh/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790607004819300-15303-0/minicluster-data/master-2/wal/instance
I20260928 14:55:59.184029 15303 external_mini_cluster.cc:1428] Setting key d1a052fb22a0381d108e02afdeac9e2d
I20260928 14:55:59.191628 5892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:55:59 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
Sep 28 14:55:59 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607359, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:59.207438 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.185679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49628 (local address 127.14.241.252:45353)
0928 14:55:59.186398 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.186409 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.186446 (+ 37us) server_negotiation.cc:407] Connection header received
0928 14:55:59.188677 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.188700 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.189114 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.189453 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.191103 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.192409 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.194046 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.195089 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.196079 (+ 990us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.199940 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:59.199951 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:59.199964 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:59.200013 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:59.203091 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.203773 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.203780 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.203786 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.203955 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.204337 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.204342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.204346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.204851 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:59.205117 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:59.205606 (+ 489us) server_negotiation.cc:299] Negotiation successful
0928 14:55:59.206052 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":310,"threads_started":1}
I20260928 14:55:59.210507 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.194406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.254:38733 (local address 127.0.0.1:49660)
0928 14:55:59.195165 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:59.195206 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:59.195423 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:59.195838 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:59.195845 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:59.196703 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:59.197016 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:59.197028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.198434 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.198439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:59.199986 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:59.199998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.201094 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.201103 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:59.201323 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.202048 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:59.202070 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:59.204243 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:59.207749 (+ 3506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:59.207761 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:59.207769 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:59.208362 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:59.208716 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:59.208721 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:59.208724 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:59.208891 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:59.209564 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:59.209570 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:59.209718 (+ 148us) client_negotiation.cc:769] Sending connection context
0928 14:55:59.209853 (+ 135us) client_negotiation.cc:240] Negotiation successful
0928 14:55:59.210140 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":369,"threads_started":1}
I20260928 14:55:59.210566 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.194375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.14.241.254:38733)
0928 14:55:59.194556 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.194562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.195296 (+ 734us) server_negotiation.cc:407] Connection header received
0928 14:55:59.195565 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.195572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.195652 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.195743 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.197180 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.198254 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.200158 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.200868 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.200986 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.204441 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:59.204446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:59.204449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:59.204484 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:59.207605 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.208470 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.208474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.208476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.208613 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.209045 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.209050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.209053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.209447 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:59.209603 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:59.210012 (+ 409us) server_negotiation.cc:299] Negotiation successful
0928 14:55:59.210234 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:55:59.211788 5892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:55:59 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607359, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
Sep 28 14:55:59 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.254@KRBTEST.COM
I20260928 14:55:59.225121 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.212503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.14.241.253:33255)
0928 14:55:59.212659 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.212665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.212805 (+ 140us) server_negotiation.cc:407] Connection header received
0928 14:55:59.212934 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.212938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.213000 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.213068 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.213851 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.214852 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.216398 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.216941 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.217069 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.219878 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:59.219885 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:59.219889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:59.219935 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:59.222536 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.223224 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.223228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.223230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.223355 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.223714 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.223719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.223722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.224080 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:59.224228 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:59.224628 (+ 400us) server_negotiation.cc:299] Negotiation successful
0928 14:55:59.224858 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:55:59.225116 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.212552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.253:33255 (local address 127.0.0.1:38504)
0928 14:55:59.212708 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:59.212724 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:59.212829 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:59.213113 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:59.213118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:59.213566 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:59.213728 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:59.213738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.214992 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.214996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:59.216225 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:59.216239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.217082 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.217086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:59.217227 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.217690 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:59.217706 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:59.219693 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:59.222656 (+ 2963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:59.222662 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:59.222666 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:59.223124 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:59.223447 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:59.223451 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:59.223455 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:59.223616 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:59.224177 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:59.224180 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:59.224373 (+ 193us) client_negotiation.cc:769] Sending connection context
0928 14:55:59.224519 (+ 146us) client_negotiation.cc:240] Negotiation successful
0928 14:55:59.224788 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 14:55:59.226286 5892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:55:59.226567 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.211205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49674 (local address 127.14.241.254:38733)
0928 14:55:59.211343 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.211347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.211359 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:55:59.211395 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.211398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.211447 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.211514 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.212830 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.213863 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.215394 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.216014 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.216135 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.220633 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:59.220640 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:59.220645 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:59.220682 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:59.223499 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.224264 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.224270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.224273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.224431 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.224871 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.224876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.224880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.225221 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:59.225479 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:59.226001 (+ 522us) server_negotiation.cc:299] Negotiation successful
0928 14:55:59.226208 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Sep 28 14:55:59 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607359, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.252@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
Sep 28 14:55:59 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.241.253@KRBTEST.COM
I20260928 14:55:59.239464 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.226963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49632)
0928 14:55:59.227119 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:59.227136 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:59.227223 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:59.227602 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:59.227607 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:59.228018 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:59.228186 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:59.228197 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.229294 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.229301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:59.230350 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:59.230362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.231108 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.231115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:59.231341 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.231865 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:59.231887 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:59.234148 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:59.237149 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:59.237153 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:59.237155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:59.237517 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:59.237832 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:59.237837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:59.237841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:59.237990 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:59.238573 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:59.238576 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:59.238711 (+ 135us) client_negotiation.cc:769] Sending connection context
0928 14:55:59.238824 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 14:55:59.239134 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260928 14:55:59.239763 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.227168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49632 (local address 127.14.241.252:45353)
0928 14:55:59.227336 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.227341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.227358 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:55:59.227403 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.227408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.227477 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.227565 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.228339 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.229165 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.230475 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.230953 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.231041 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.234319 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:59.234323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:59.234327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:59.234354 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:59.237022 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.237612 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.237617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.237620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.237726 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.238083 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.238088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.238091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.238456 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:59.238594 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:59.238930 (+ 336us) server_negotiation.cc:299] Negotiation successful
0928 14:55:59.239285 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":49}
I20260928 14:55:59.242996 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.228199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38516 (local address 127.14.241.253:33255)
0928 14:55:59.228382 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.228387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.228405 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:55:59.228485 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.228489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.228556 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.228647 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.229845 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.230564 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.231938 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.232387 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.232477 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.237404 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:59.237409 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:59.237412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:59.237444 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:59.240598 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.241331 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.241335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.241338 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.241478 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.241862 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.241865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.241867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.242110 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:59.242327 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:59.242599 (+ 272us) server_negotiation.cc:299] Negotiation successful
0928 14:55:59.242764 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 14:55:59.244992 5892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d: Bootstrap starting.
I20260928 14:55:59.247198 15303 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:55:59.248031 5892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:59.249078 5892 log.cc:824] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:59.252199 5892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d: No bootstrap required, opened a new log
I20260928 14:55:59.249533 5712 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "44c9c4a027b44dc58545fd5343e803a6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49674
I20260928 14:55:59.262723 5892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } }
I20260928 14:55:59.263139 5892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ab76abb58484dd591a7f740007f700d, State: Initialized, Role: FOLLOWER
I20260928 14:55:59.263651 5892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } }
I20260928 14:55:59.264591 5898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } } }
I20260928 14:55:59.264911 5898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:59.265661 5892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [sys.catalog]: configured and running, proceeding with master startup.
W20260928 14:55:59.275452 5909 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:55:59.275609 5909 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:55:59.273254 5822 master_runner.cc:186] Error getting master registration for 127.14.241.254:38733: OK, instance_id { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" instance_seqno: 1790607358188406 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:55:59.281406 5822 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:55:59.323630 5737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } attempt: 4
Sep 28 14:55:59 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607358, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.254@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
I20260928 14:55:59.338486 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.324393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49636)
0928 14:55:59.324576 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:59.324597 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:59.324743 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:59.325102 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:59.325107 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:59.325575 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:59.325782 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:59.325796 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.326851 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.326856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:59.327799 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:59.327812 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.328567 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.328572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:59.328773 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.329355 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:59.329371 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:59.331756 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:59.335699 (+ 3943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:59.335705 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:59.335709 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:59.336239 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:59.336637 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:59.336642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:59.336646 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:59.336831 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:59.337468 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:59.337473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:59.337664 (+ 191us) client_negotiation.cc:769] Sending connection context
0928 14:55:59.337818 (+ 154us) client_negotiation.cc:240] Negotiation successful
0928 14:55:59.338100 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260928 14:55:59.338517 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.324575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49636 (local address 127.14.241.252:45353)
0928 14:55:59.324806 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.324811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.324832 (+ 21us) server_negotiation.cc:407] Connection header received
0928 14:55:59.324897 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.324901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.324967 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.325049 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.325929 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.326706 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.327936 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.328433 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.328548 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.331992 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:59.331999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:59.332004 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:59.332046 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:59.335556 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.336388 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.336393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.336398 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.336535 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.336935 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.336940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.336944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.337345 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:59.337487 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:59.337936 (+ 449us) server_negotiation.cc:299] Negotiation successful
0928 14:55:59.338166 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:55:59.344708 5737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6: Bootstrap starting.
I20260928 14:55:59.348817 5737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:59.350142 5737 log.cc:824] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:59.353412 5737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6: No bootstrap required, opened a new log
I20260928 14:55:59.360978 5737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } }
I20260928 14:55:59.361384 5737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c9c4a027b44dc58545fd5343e803a6, State: Initialized, Role: FOLLOWER
I20260928 14:55:59.361909 5737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } }
I20260928 14:55:59.362914 5912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } } }
I20260928 14:55:59.363274 5912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:59.364100 5737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:59.364156 5712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 14:55:59.364355 5712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:59.367118 5712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } }
I20260928 14:55:59.369407 5867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c9c4a027b44dc58545fd5343e803a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8ab76abb58484dd591a7f740007f700d"
I20260928 14:55:59.369894 5867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:59.373366 5867 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44c9c4a027b44dc58545fd5343e803a6 in term 1.
I20260928 14:55:59.374058 5686 leader_election.cc:302] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44c9c4a027b44dc58545fd5343e803a6, 8ab76abb58484dd591a7f740007f700d; no voters:
I20260928 14:55:59.374528 5912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:55:59.375211 5912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [term 1 LEADER]: Becoming Leader. State: Replica: 44c9c4a027b44dc58545fd5343e803a6, State: Running, Role: LEADER
I20260928 14:55:59.375789 5912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } }
I20260928 14:55:59.374505 5712 leader_election.cc:288] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4a903636b5e5422e9c140129df6e45f0 (127.14.241.253:33255), 8ab76abb58484dd591a7f740007f700d (127.14.241.252:45353)
I20260928 14:55:59.380404 5915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c9c4a027b44dc58545fd5343e803a6. Latest consensus state: current_term: 1 leader_uuid: "44c9c4a027b44dc58545fd5343e803a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } } }
I20260928 14:55:59.381219 5915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:59.390460 5918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:55:59.392338 5815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } attempt: 5
Sep 28 14:55:59 dist-test-slave-vk40 krb5kdc[5657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790607358, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.241.253@KRBTEST.COM for kudu/127.14.241.252@KRBTEST.COM
W20260928 14:55:59.413307 5845 tablet.cc:2369] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:55:59.428434 5691 tablet.cc:2369] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:55:59.428723 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.393912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.14.241.252:45353)
0928 14:55:59.394083 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.394089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.394107 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:55:59.394182 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.394187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.394275 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.394380 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.395672 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.396963 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.398958 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.399752 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.399876 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.405985 (+ 6109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:59.405993 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:59.405999 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:59.406045 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:59.411002 (+ 4957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.412481 (+ 1479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.412513 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.412518 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.412816 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.414902 (+ 2086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.414907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.414910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.421773 (+ 6863us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:59.422767 (+ 994us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:59.423734 (+ 967us) server_negotiation.cc:299] Negotiation successful
0928 14:55:59.425616 (+ 1882us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:55:59.429348 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.393079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.241.252:45353 (local address 127.0.0.1:49640)
0928 14:55:59.393528 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0928 14:55:59.393551 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 14:55:59.393693 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:55:59.394640 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:55:59.394647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:55:59.395200 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:55:59.395424 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:59.395437 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.397170 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.397177 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:59.398737 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:55:59.398754 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.400152 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.400181 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:55:59.400512 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.401376 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:55:59.401421 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:55:59.405641 (+ 4220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:55:59.411281 (+ 5640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:59.411287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:59.411293 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:59.412263 (+ 970us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:59.414344 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:55:59.414350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:55:59.414353 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:55:59.414724 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:55:59.422222 (+ 7498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:55:59.422229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:55:59.422815 (+ 586us) client_negotiation.cc:769] Sending connection context
0928 14:55:59.423496 (+ 681us) client_negotiation.cc:240] Negotiation successful
0928 14:55:59.426171 (+ 2675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307}
I20260928 14:55:59.435961 5815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0: Bootstrap starting.
I20260928 14:55:59.441848 5815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0: Neither blocks nor log segments found. Creating new log.
I20260928 14:55:59.447762 5815 log.cc:824] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0: Log is configured to *not* fsync() on all Append() calls
I20260928 14:55:59.460196 5815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0: No bootstrap required, opened a new log
I20260928 14:55:59.465878 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.408604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49686 (local address 127.14.241.254:38733)
0928 14:55:59.408820 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.408828 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.430441 (+ 21613us) server_negotiation.cc:407] Connection header received
0928 14:55:59.430642 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.430649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.430746 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.430862 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.442631 (+ 11769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.443822 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.446010 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.446761 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.446867 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.450321 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:59.450328 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:59.450333 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:59.450381 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:59.454456 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.455178 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.455184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.455187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.455366 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.455764 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.455769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.455772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.456128 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:59.456412 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:59.457210 (+ 798us) server_negotiation.cc:299] Negotiation successful
0928 14:55:59.457444 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 14:55:59.456266 5918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:55:59.470990 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.415197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.14.241.252:45353)
0928 14:55:59.443090 (+ 27893us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.443097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.443289 (+ 192us) server_negotiation.cc:407] Connection header received
0928 14:55:59.443864 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.443872 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.444671 (+ 799us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.445479 (+ 808us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.447424 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.448416 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.450409 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.451072 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.454899 (+ 3827us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.455098 (+ 199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:59.455103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:59.455107 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:59.455167 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:59.467191 (+ 12024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.468069 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.468074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.468078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.468256 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.468734 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.468739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.468742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.469110 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:59.470181 (+ 1071us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:59.470326 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 14:55:59.470578 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26843,"thread_start_us":50110,"threads_started":1}
I20260928 14:55:59.488634 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:55:59.428869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38532 (local address 127.14.241.253:33255)
0928 14:55:59.429057 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.429063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.456948 (+ 27885us) server_negotiation.cc:407] Connection header received
0928 14:55:59.457446 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.457451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.457531 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.457636 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.459268 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.460354 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.462398 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:55:59.463035 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:55:59.463149 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:55:59.466537 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:55:59.466543 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:55:59.466548 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:55:59.466586 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:55:59.470244 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.471783 (+ 1539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.471788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.471792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.471952 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:55:59.486367 (+ 14415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:55:59.486374 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:55:59.486378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:55:59.486907 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:55:59.487180 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0928 14:55:59.487879 (+ 699us) server_negotiation.cc:299] Negotiation successful
0928 14:55:59.488182 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:55:59.475834 5670 master_runner.cc:186] Error getting master registration for 127.14.241.253:33255: OK, instance_id { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" instance_seqno: 1790607358671320 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:55:59.489645 5670 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:55:59.490742 5815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } }
I20260928 14:55:59.491137 5815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a903636b5e5422e9c140129df6e45f0, State: Initialized, Role: FOLLOWER
I20260928 14:55:59.491672 5815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } }
I20260928 14:55:59.500855 5938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } } }
I20260928 14:55:59.503082 5938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:59.502995 5788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44c9c4a027b44dc58545fd5343e803a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4a903636b5e5422e9c140129df6e45f0"
I20260928 14:55:59.503492 5788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:55:59.502948 5815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:55:59.506891 5788 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44c9c4a027b44dc58545fd5343e803a6 in term 1.
I20260928 14:55:59.508677 5867 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [term 1 FOLLOWER]: Refusing update from remote peer 44c9c4a027b44dc58545fd5343e803a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:59.509938 5915 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:59.510251 5787 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [term 1 FOLLOWER]: Refusing update from remote peer 44c9c4a027b44dc58545fd5343e803a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:55:59.511032 5915 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:55:59.536242 5898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c9c4a027b44dc58545fd5343e803a6. Latest consensus state: current_term: 1 leader_uuid: "44c9c4a027b44dc58545fd5343e803a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } } }
I20260928 14:55:59.536660 5898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:59.539526 5938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c9c4a027b44dc58545fd5343e803a6. Latest consensus state: current_term: 1 leader_uuid: "44c9c4a027b44dc58545fd5343e803a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } } }
I20260928 14:55:59.539783 5938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:59.544677 5942 mvcc.cc:205] Tried to move back new op lower bound from 7334327744534958080 to 7334327744008355840. Current Snapshot: MvccSnapshot[applied={T|T < 7334327744534958080}]
I20260928 14:55:59.544941 5943 mvcc.cc:205] Tried to move back new op lower bound from 7334327744534958080 to 7334327744008355840. Current Snapshot: MvccSnapshot[applied={T|T < 7334327744534958080}]
I20260928 14:55:59.550355 5938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44c9c4a027b44dc58545fd5343e803a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } } }
I20260928 14:55:59.550612 5938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:59.553952 5915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44c9c4a027b44dc58545fd5343e803a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } } }
I20260928 14:55:59.554245 5915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:59.554560 5915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44c9c4a027b44dc58545fd5343e803a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } } }
I20260928 14:55:59.554783 5915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6 [sys.catalog]: This master's current role is: LEADER
I20260928 14:55:59.555421 5918 catalog_manager.cc:1347] Generated new cluster ID: 46eb50689ddb430a8e12e77d41e474ad
I20260928 14:55:59.555524 5918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:55:59.555907 5898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44c9c4a027b44dc58545fd5343e803a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c9c4a027b44dc58545fd5343e803a6" member_type: VOTER last_known_addr { host: "127.14.241.254" port: 38733 } } peers { permanent_uuid: "4a903636b5e5422e9c140129df6e45f0" member_type: VOTER last_known_addr { host: "127.14.241.253" port: 33255 } } peers { permanent_uuid: "8ab76abb58484dd591a7f740007f700d" member_type: VOTER last_known_addr { host: "127.14.241.252" port: 45353 } } }
I20260928 14:55:59.556185 5898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:55:59.587751 5918 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:55:59.589933 5918 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:55:59.612717 5956 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 14:55:59.625939 5918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44c9c4a027b44dc58545fd5343e803a6: Generated new TSK 0
I20260928 14:55:59.627064 5918 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:55:59.723002 5741 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:59.710029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49690 (local address 127.14.241.254:38733)
0928 14:55:59.710258 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.710266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.720931 (+ 10665us) server_negotiation.cc:407] Connection header received
0928 14:55:59.721258 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.721265 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.721374 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.721453 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.722729 (+ 1276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49690: BlockingRecv error: recv got EOF from 127.0.0.1:49690 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260928 14:55:59.723464 5895 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:59.710234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.14.241.252:45353)
0928 14:55:59.710403 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.710410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.721176 (+ 10766us) server_negotiation.cc:407] Connection header received
0928 14:55:59.721405 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.721410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.721490 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.721554 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.723207 (+ 1653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49664: BlockingRecv error: recv got EOF from 127.0.0.1:49664 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260928 14:55:59.725769 5819 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:59.716318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38542 (local address 127.14.241.253:33255)
0928 14:55:59.716523 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.716529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.722563 (+ 6034us) server_negotiation.cc:407] Connection header received
0928 14:55:59.724325 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.724331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.724417 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.724485 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.725520 (+ 1035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38542: BlockingRecv error: recv got EOF from 127.0.0.1:38542 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260928 14:55:59.731618 5819 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:59.729750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38554 (local address 127.14.241.253:33255)
0928 14:55:59.729930 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.729936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.730378 (+ 442us) server_negotiation.cc:407] Connection header received
0928 14:55:59.730580 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.730584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.730684 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.730767 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.731434 (+ 667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38554: BlockingRecv error: recv got EOF from 127.0.0.1:38554 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260928 14:55:59.737679 5819 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:59.735951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38564 (local address 127.14.241.253:33255)
0928 14:55:59.736125 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.736131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.736599 (+ 468us) server_negotiation.cc:407] Connection header received
0928 14:55:59.736773 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.736776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.736843 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.736924 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.737538 (+ 614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38564: BlockingRecv error: recv got EOF from 127.0.0.1:38564 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260928 14:55:59.744882 5819 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:59.742934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.14.241.253:33255)
0928 14:55:59.743118 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.743125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.743602 (+ 477us) server_negotiation.cc:407] Connection header received
0928 14:55:59.743778 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.743784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.743854 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.743936 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.744659 (+ 723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38572: BlockingRecv error: recv got EOF from 127.0.0.1:38572 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260928 14:55:59.753513 5819 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:59.751506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38580 (local address 127.14.241.253:33255)
0928 14:55:59.751718 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.751725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.752227 (+ 502us) server_negotiation.cc:407] Connection header received
0928 14:55:59.752400 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.752405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.752488 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.752573 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.753346 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38580: BlockingRecv error: recv got EOF from 127.0.0.1:38580 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260928 14:55:59.766698 5819 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:59.764721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38590 (local address 127.14.241.253:33255)
0928 14:55:59.764919 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.764926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.765393 (+ 467us) server_negotiation.cc:407] Connection header received
0928 14:55:59.765603 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.765609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.765690 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.765782 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.766464 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38590: BlockingRecv error: recv got EOF from 127.0.0.1:38590 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260928 14:55:59.787617 5819 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:59.785546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.14.241.253:33255)
0928 14:55:59.785748 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.785755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.786293 (+ 538us) server_negotiation.cc:407] Connection header received
0928 14:55:59.786486 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.786490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.786567 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.786665 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.787381 (+ 716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38604: BlockingRecv error: recv got EOF from 127.0.0.1:38604 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260928 14:55:59.824484 5819 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:59.822603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38606 (local address 127.14.241.253:33255)
0928 14:55:59.822804 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.822812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.823092 (+ 280us) server_negotiation.cc:407] Connection header received
0928 14:55:59.823287 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.823292 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.823370 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.823455 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.824189 (+ 734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38606: BlockingRecv error: recv got EOF from 127.0.0.1:38606 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260928 14:55:59.893486 5819 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:55:59.891651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38608 (local address 127.14.241.253:33255)
0928 14:55:59.891868 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 14:55:59.891875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:55:59.892145 (+ 270us) server_negotiation.cc:407] Connection header received
0928 14:55:59.892341 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:55:59.892348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:55:59.892448 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:55:59.892573 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:55:59.893248 (+ 675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38608: BlockingRecv error: recv got EOF from 127.0.0.1:38608 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260928 14:56:00.026548 5819 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:56:00.024592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.14.241.253:33255)
0928 14:56:00.024841 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0928 14:56:00.024849 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:56:00.025162 (+ 313us) server_negotiation.cc:407] Connection header received
0928 14:56:00.025363 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:56:00.025367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:56:00.025443 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:56:00.025528 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:56:00.026303 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38624: BlockingRecv error: recv got EOF from 127.0.0.1:38624 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 14:56:00.277715 5909 catalog_manager.cc:1259] Loaded cluster ID: 46eb50689ddb430a8e12e77d41e474ad
I20260928 14:56:00.277827 5909 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d: loading cluster ID for follower catalog manager: success
I20260928 14:56:00.280437 5909 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d: acquiring CA information for follower catalog manager: success
I20260928 14:56:00.282371 5909 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8ab76abb58484dd591a7f740007f700d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260928 14:56:00.287602 5819 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:56:00.285905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38626 (local address 127.14.241.253:33255)
0928 14:56:00.286077 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 14:56:00.286082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:56:00.286291 (+ 209us) server_negotiation.cc:407] Connection header received
0928 14:56:00.286529 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:56:00.286534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:56:00.286628 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:56:00.286732 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:56:00.287431 (+ 699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38626: BlockingRecv error: recv got EOF from 127.0.0.1:38626 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:56:00.617082 5956 catalog_manager.cc:1259] Loaded cluster ID: 46eb50689ddb430a8e12e77d41e474ad
I20260928 14:56:00.617192 5956 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0: loading cluster ID for follower catalog manager: success
I20260928 14:56:00.619899 5956 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0: acquiring CA information for follower catalog manager: success
I20260928 14:56:00.621858 5956 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4a903636b5e5422e9c140129df6e45f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:56:00.813395 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:56:00.805912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38630 (local address 127.14.241.253:33255)
0928 14:56:00.806787 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0928 14:56:00.806794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:56:00.807109 (+ 315us) server_negotiation.cc:407] Connection header received
0928 14:56:00.807344 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:56:00.807348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:56:00.807494 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:56:00.807602 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:56:00.808157 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:56:00.809051 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:56:00.810936 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:56:00.811377 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:56:00.811471 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:56:00.812112 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:56:00.812745 (+ 633us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:56:00.812870 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0928 14:56:00.812986 (+ 116us) server_negotiation.cc:299] Negotiation successful
0928 14:56:00.813054 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":509,"threads_started":1}
I20260928 14:56:00.825647 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:56:00.820003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49666 (local address 127.14.241.252:45353)
0928 14:56:00.820631 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0928 14:56:00.820637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:56:00.820654 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:56:00.820749 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:56:00.820753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:56:00.820914 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:56:00.821055 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:56:00.821463 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:56:00.822170 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:56:00.823443 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:56:00.823846 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:56:00.823953 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:56:00.824502 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:56:00.825058 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:56:00.825194 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0928 14:56:00.825276 (+ 82us) server_negotiation.cc:299] Negotiation successful
0928 14:56:00.825360 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":393,"threads_started":1}
I20260928 14:56:00.850745 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:56:00.831701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49668 (local address 127.14.241.252:45353)
0928 14:56:00.837302 (+ 5601us) server_negotiation.cc:206] Beginning negotiation
0928 14:56:00.837308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:56:00.843340 (+ 6032us) server_negotiation.cc:407] Connection header received
0928 14:56:00.843758 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:56:00.843765 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:56:00.843876 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:56:00.843944 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:56:00.844764 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:56:00.845985 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:56:00.848270 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:56:00.849007 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:56:00.849117 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:56:00.849845 (+ 728us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:56:00.849999 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:56:00.850058 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:56:00.850303 (+ 245us) server_negotiation.cc:299] Negotiation successful
0928 14:56:00.850404 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5431}
I20260928 14:56:00.858058 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:56:00.837690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38644 (local address 127.14.241.253:33255)
0928 14:56:00.837873 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 14:56:00.837878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:56:00.851974 (+ 14096us) server_negotiation.cc:407] Connection header received
0928 14:56:00.852235 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:56:00.852241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:56:00.852366 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:56:00.852476 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:56:00.853005 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:56:00.853972 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:56:00.855852 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:56:00.856560 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:56:00.856674 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:56:00.857034 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:56:00.857170 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:56:00.857267 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 14:56:00.857453 (+ 186us) server_negotiation.cc:299] Negotiation successful
0928 14:56:00.857550 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 14:56:00.858930 6036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:56:00.831240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49696 (local address 127.14.241.254:38733)
0928 14:56:00.852839 (+ 21599us) server_negotiation.cc:206] Beginning negotiation
0928 14:56:00.852844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:56:00.852863 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:56:00.852964 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:56:00.852968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:56:00.853189 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:56:00.853380 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:56:00.853884 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:56:00.854643 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:56:00.856232 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:56:00.856910 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:56:00.857078 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:56:00.857644 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:56:00.858119 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:56:00.858595 (+ 476us) server_negotiation.cc:1035] Waiting for connection context
0928 14:56:00.858637 (+ 42us) server_negotiation.cc:299] Negotiation successful
0928 14:56:00.858705 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21480,"thread_start_us":21419,"threads_started":1}
I20260928 14:56:00.863942 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 5670
I20260928 14:56:00.865226 5732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:56:01.008510 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 5670
W20260928 14:56:01.025713 5763 connection.cc:535] server connection from 127.0.0.1:38494 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 14:56:01.026041 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 5744
I20260928 14:56:01.027096 5809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:56:01.135473 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 5744
I20260928 14:56:01.153333 15303 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKMueFh/build/asan/bin/kudu pid 5822
I20260928 14:56:01.154425 5887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260928 14:56:01.263197 15303 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKMueFh/build/asan/bin/kudu with pid 5822
2026-09-28T14:56:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4762 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9519 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (356399 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST